BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-25 10:43:49 +00:00
Code Releases Wiki Activity
6,099 Commits 39 Branches 119 Tags
Commit Graph

3904 Commits

Author SHA1 Message Date
Paul Tremberth
d35a01a103 Update default password 2017-02-20 14:23:23 +01:00
Paul Tremberth
565baae1e2 Change FTP_* settings names 2017-02-20 14:19:36 +01:00
Paul Tremberth
3ffa2c5759 Support Anonymous FTP 
Fixes GH-2342
 2017-02-20 14:19:36 +01:00
Mikhail Korobov
afac3fd2c2 Merge pull request #2530 from elacuesta/proxy_credentials 
[MRG+1] Use credentials from request.meta['proxy']
 2017-02-14 20:58:39 +06:00
Eugenio Lacuesta
ae0ea31abd Add HTTPPROXY_ENABLED setting (default True) 2017-02-14 11:33:01 -03:00
Daniel Graña
7dd7646e65 Bump version: 1.3.1 → 1.3.2 2017-02-13 14:57:55 -03:00
Mikhail Korobov
1bc4d8b6b6 fixed tls in Twisted 17+ 2017-02-13 20:03:53 +05:00
Paul Tremberth
f32a229e3d Merge pull request #2510 from elacuesta/reqser_request_class 
[MRG+1] Preserve request class when converting to/from dicts (utils.reqser)
 2017-02-08 18:30:07 +01:00
Eugenio Lacuesta
9c0aae724e Use credentials from request.meta['proxy'] if present 2017-02-08 13:22:30 -03:00
Paul Tremberth
af55a8713e Bump version: 1.3.0 → 1.3.1 2017-02-08 17:08:19 +01:00
Paul Tremberth
a8893190a1 Merge pull request #2464 from elacuesta/component_order_integer 
[MRG+2] Validate numeric values (components order)
 2017-02-08 13:01:02 +01:00
Paul Tremberth
3becb861af Merge pull request #2544 from scrapy/twisted-version-cleanup 
[MRG+1] Remove code required to support ancient twisted versions.
 2017-02-08 12:59:45 +01:00
Paul Tremberth
e87f18447c Merge pull request #2538 from scrapy/linkextractor-fail-fast 
LinkExtractor: don’t check all regexes if one of them matches
 2017-02-08 12:58:41 +01:00
Rolando Espinoza
4e765acaed BUG: Fix __classcell__ propagation. 
Python 3.6 added simpler customization of class creation but this
requires to propagate correctly the __classcell__ attribute in custom
__new__ methods.

See https://docs.python.org/3.6/whatsnew/3.6.html#pep-487-simpler-
customization-of-class-creation
 2017-02-08 02:12:52 -04:00
Mikhail Korobov
04b2f79e7a Remove code required to support ancient twisted versions. See GH-1887. 2017-02-07 22:30:58 +05:00
Mikhail Korobov
eaf62ab69c cleanup MetaRefreshMiddleware: remove redundant check 2017-02-07 17:56:43 +05:00
Eugenio Lacuesta
24e82bfe75 Validate values for components order 2017-02-07 09:13:57 -03:00
Mikhail Korobov
f73eb715ac LinkExtractor: don’t check all regexes if one of them matches 2017-02-07 02:16:06 +05:00
Paul Tremberth
0cf6344cc2 Support kwargs for response.xpath() 2017-02-02 17:44:57 +01:00
Mikhail Korobov
c305c46254 Merge pull request #2503 from redapple/view-no-redirect 
[MRG] Fix view command against new --no-redirect option
 2017-02-02 21:24:56 +05:00
Mikhail Korobov
d4a8f43816 Merge pull request #2519 from redapple/settings-getbool-new 
[MRG+1] Support 'True' and 'False' strings as boolean settings values
 2017-02-02 21:15:24 +05:00
Mikhail Korobov
245287fd5a Merge pull request #2496 from redapple/dns-resolver-timeout 
[MRG] Enforce DNS resolution timeout
 2017-02-02 21:13:59 +05:00
Mikhail Korobov
9c75403176 Merge pull request #2466 from eLRuLL/settings-pipeline-template 
[MRG+1] settings: fixing name of the pipeline template
 2017-02-02 17:16:57 +05:00
Paul Tremberth
d2e9ea0c88 Enforce DNS resolution timeout 2017-01-31 18:21:11 +01:00
Paul Tremberth
ae6d8d728e Support 'True' and 'False' strings as boolean settings values 2017-01-30 16:33:08 +01:00
Paul Tremberth
fc07711614 Remove unused --headers option for view command 2017-01-30 15:54:28 +01:00
Eugenio Lacuesta
53757e51e5 Preserve request class when converting to/from dicts (utils.reqser) 2017-01-24 17:13:21 -03:00
Paul Tremberth
b279bc8546 Fix view command against new --no-redirect option 2017-01-19 16:28:52 +01:00
Paul Tremberth
e7dede66f6 Merge pull request #2458 from rolando-contrib/http11-maxsize-logging-tweak 
[MRG+1] ENH Pass arguments to logger rather than formatted message.
 2017-01-03 14:01:34 +01:00
Raul Gallegos
e7c7e055ff settings: fixing name of the pipeline template 2016-12-24 11:55:04 -05:00
Paul Tremberth
ac74d5a467 Bump version: 1.2.2 → 1.3.0 2016-12-21 16:28:44 +01:00
Elias Dorneles
d09ec3db68 Merge pull request #2410 from redapple/fetch-transparent-redirect 
[MRG+1] Transparently handle redirections in fetch and shell
 2016-12-21 09:49:15 -02:00
Rolando Espinoza
6dec4a3ccb ENH Pass arguments to logger rather than formatted message. 
This not only use the standard form but helps error aggregation
libraries (i.e.: Sentry) to avoid duplicating the message.
 2016-12-20 20:02:31 -04:00
Mikhail Korobov
d19c4c1f80 Merge pull request #2433 from redapple/wrong-spidermodules-warning 
[MRG+1] Warn user instead of failing for wrong SPIDER_MODULES setting
 2016-12-19 21:46:56 +05:00
Paul Tremberth
ed1e4d8df3 Merge pull request #1731 from scrapy/disable-toplevel-2 
[MRG+1] LOG_SHORT_NAMES option to disable TopLevelFormatter
 2016-12-19 16:02:55 +01:00
Elias Dorneles
97e82107b1 Merge pull request #2270 from redapple/httperror-log-info 
[MRG+1] Raise log level for HttpErrorMiddleware to INFO (from DEBUG)
 2016-12-14 12:18:04 -02:00
Paul Tremberth
70a69d2199 Use built-in range() 2016-12-12 22:40:48 +01:00
Paul Tremberth
f457379a54 Add stacktrace in warning message 2016-12-09 16:56:26 +01:00
Akhil Lb
a75ad2bbc6 LOG_SHORT_NAMES option 2016-12-09 00:04:04 +05:00
Paul Tremberth
7d17836032 Update documentation about --no-redirect option 2016-12-08 17:27:25 +01:00
Paul Tremberth
948e3cd003 Warn user instead of failing for wrong SPIDER_MODULES setting 2016-12-08 12:50:26 +01:00
Mikhail Korobov
bd8c293a97 Merge pull request #2411 from redapple/remove-chunked-transfer-mw 
[MRG+1] Remove ChunkedTransferMiddleware from default settings
 2016-12-07 22:05:15 +05:00
Elias Dorneles
cce631abec Merge pull request #1887 from nyov/twisted11 
[MRG+2] Bump Twisted dependency to 13.1.0
 2016-12-07 15:00:01 -02:00
Paul Tremberth
778bed07bf Let framework handle only HTTP redirects by default for fetch and shell commands 2016-12-07 17:56:13 +01:00
Mikhail Korobov
ff3aec6613 Merge pull request #2331 from moisesguimaraes/fixes-2272 
[MRG+1] Fixes issue #2272 using arg_to_iter() to wrap single values and list() to…
 2016-12-07 20:08:18 +05:00
Paul Tremberth
f3d5995329 Bump version: 1.2.1 → 1.2.2 2016-12-06 15:21:00 +01:00
nyov
67bc2e0b18 Wipe scrapy.xlib.tx 2016-12-02 21:22:35 +00:00
nyov
67cf64edbe Deprecate scrapy.xlib.tx 2016-12-02 21:22:35 +00:00
nyov
534772f6ea Import xlib.tx code from twisted proper 2016-12-02 21:21:51 +00:00
nyov
985755d1fe Remove obsolete xlib code for Twisted 13.1.0 2016-12-01 23:44:15 +00:00