Luar Roji
ee66382f29
Fixed banner display in all 3 python shells
2015-08-21 01:12:58 +01:00
nyov
509cc8d41e
Add support for bpython console.
...
Adds support for configuration of shells from scrapy.cfg
and SCRAPY_PYTHON_SHELL.
config snippet:
cat <<EOF >> ~/.scrapy.cfg
[settings]
# shell can be one of ipython, bpython or python;
# to be tried as the interactive python console
# (in above order, unless set here).
shell = python
EOF
(closes #270 , #1100 , #1301 )
2015-08-21 01:12:58 +01:00
Aivars Kalvāns
b8b1e8e544
Make common practices sample code match the comments
2015-08-19 16:54:10 +03:00
Julia Medina
d3c3408cd4
Replace alternative OSX install workaround with virtualenv
2015-08-19 03:19:27 -03:00
Julia Medina
46ee1c6ee2
Reference Homebrew's homepage for installation instructions
2015-08-19 02:21:08 -03:00
Daniel Graña
280eab2416
Merge pull request #1352 from nyov/le-bogus-links
...
[MRG+1] [LinkExtractors] Ignore bogus links (#907 )
2015-08-16 00:42:55 -03:00
Andrew Scorpil
de15fcdf33
[LinkExtractors] Ignore bogus links
...
(rebased the code for scrapy 1.0 and made a few code improvements --nyov)
2015-08-15 00:16:39 +00:00
nyov
1b4fd3a8df
Support anonymous connections in S3DownloadHandler
...
Also consider any unknown keyword args for S3DownloadHandler as
arguments to pass on to S3Connection (e.g. proxy settings).
2015-08-14 22:12:12 +00:00
Mikhail Korobov
9adb5c31c0
Merge pull request #1433 from scrapy/codecov
...
Coverage and reports at codecov.io and coveralls.io
2015-08-14 05:21:10 +05:00
Daniel Graña
27077d2b6c
Forward coverage envvars needed to cover subprocess
2015-08-13 21:07:45 -03:00
Daniel Graña
43fc2f23b8
Increase coverage of version command
2015-08-13 19:30:06 -03:00
Daniel Graña
54f9837804
Document how to get coverage report
2015-08-13 19:02:51 -03:00
Daniel Graña
cb9577a6d4
Do not be verbose with coverage report by default
2015-08-13 19:02:36 -03:00
Daniel Graña
108c3e63a5
Add coverage report trough codecov.io
2015-08-13 13:56:24 -03:00
Daniel Graña
12bebb6172
Merge pull request #1431 from dangra/moved-to-parsel
...
Replace usage of deprecated class by its parsel\'s counterpart
2015-08-12 14:29:34 -03:00
Daniel Graña
6c2c9a4baf
Merge pull request #1430 from dangra/openssl-version
...
Add openssl version to version command
2015-08-12 14:24:43 -03:00
Daniel Graña
7f634598b6
Add openssl version to version command
2015-08-12 14:24:24 -03:00
Daniel Graña
4ce0f53f7e
Replace usage of deprecated class by its parsel\'s counterpart
2015-08-12 14:21:55 -03:00
Daniel Graña
68d2311298
Merge pull request #1427 from scrapy/python3-porting-status
...
add Python 3 porting status badge to README
2015-08-12 11:15:33 -03:00
Mikhail Korobov
1ed6a01c4b
Add Python 3 porting status badge to the README
2015-08-12 19:13:54 +05:00
Daniel Graña
15c1300d35
Merge pull request #1409 from eliasdorneles/migrate-parsel
...
[MRG+1] Migrating selectors to use parsel
2015-08-11 16:22:58 -03:00
Elias Dorneles
a5abd19e84
make Parsel's Selector more private, remove direct dependency of ParselSelectorList
2015-08-11 15:58:29 -03:00
Elias Dorneles
766c255152
upgrade parsel and add shim for deprecated selectorlist methods
2015-08-11 15:20:33 -03:00
Daniel Graña
457b97c13c
Merge pull request #1388 from scrapy/dupefilter-persist
...
Dupefilter persistence
2015-08-11 14:16:34 -03:00
Daniel Graña
e2f31f3018
explicit close file on file:// scheme handler
2015-08-11 14:09:49 -03:00
Elias Dorneles
e50610bd3a
set base_url in kwargs to be fully backward compatible
2015-08-11 14:09:34 -03:00
Elias Dorneles
8ef5aa2ffc
using bytes for response body in tests
2015-08-11 14:09:34 -03:00
Elias Dorneles
3a03ef7c08
cleanup tests for selectors and translators
2015-08-11 14:09:34 -03:00
Elias Dorneles
12579b9afa
warning when ambiguous root arguments and minor cleanups
2015-08-11 14:09:34 -03:00
Elias Dorneles
26ebccd37a
upgrade parsel and use its function to instantiate root for finding form
2015-08-11 14:09:34 -03:00
Elias Dorneles
2fe6d128f5
upgrade parsel and using promoted root attribute
2015-08-11 14:09:16 -03:00
Elias Dorneles
67c98b185b
avoid harcoded check for selector type
2015-08-11 14:09:00 -03:00
Elias Dorneles
94c3a345b7
remove deprecated module lxmldocument
2015-08-11 14:09:00 -03:00
Elias Dorneles
6287fc3109
remove lxmldocument dependency from http.request.form
2015-08-11 14:09:00 -03:00
Elias Dorneles
35c1dcdbc2
use response.selector in link extractors instead of instantiating new Selector
2015-08-11 14:09:00 -03:00
Elias Dorneles
17d7347a36
update minimal parsel version, add deprecated classes for csstranslator modules, fix test
2015-08-11 14:09:00 -03:00
Elias Dorneles
01d948f0fd
remove selector support for LxmlDocument DOM cache and add deprecation warning for _root argument
2015-08-11 14:08:43 -03:00
Elias Dorneles
3a572e2f3b
cleanup csstranslator module, keeping only imports
2015-08-11 14:08:43 -03:00
Elias Dorneles
c7b29d118d
fix support to legacy _root argument
2015-08-11 14:08:43 -03:00
Elias Dorneles
ce21884a97
migrating scrapy Selector to use Parsel
2015-08-11 14:08:43 -03:00
Daniel Graña
f67a808d9e
Add 1.0.3 release notes
2015-08-11 13:49:49 -03:00
Daniel Graña
342c29af5d
Merge pull request #1425 from scrapy/service-identity
...
add service_identity to scrapy install_requires. Fix GH-1418 and GH-850.
2015-08-11 13:34:36 -03:00
Mikhail Korobov
75d3d03382
add service_identity to scrapy install_requires
2015-08-11 20:14:22 +05:00
Mikhail Korobov
fa123b335b
Merge pull request #1421 from dangra/nyov/lazyload-downloadhandlers
...
lazy-load downloadhandlers (continuation of #1357 )
2015-08-11 18:59:47 +05:00
Daniel Graña
7717501ab2
Use log formatting and pass crawler reference
2015-08-11 10:38:31 -03:00
Mikhail Korobov
fa3d84b050
Merge pull request #1420 from scrapy/py3-trackrefs
...
PY3: port scrapy.utils.trackref
2015-08-11 18:21:34 +05:00
Daniel Graña
8738521d0a
Add docstrings
2015-08-10 19:35:57 -03:00
Daniel Graña
eb44152a58
lints
2015-08-10 18:25:11 -03:00
Daniel Graña
15ccf79cad
Log errors importing or initializing download handlers
2015-08-10 18:18:13 -03:00
Daniel Graña
d3804b3439
log errors importing or instanciating handlers
2015-08-10 16:52:44 -03:00
