BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-27 04:24:05 +00:00
Code Releases Wiki Activity
5,472 Commits 39 Branches 119 Tags
Commit Graph

5472 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Konstantin Lopuhin
a5da7531c4 py3 backout skipping test_closespider - it was fixed on another branch 2016-01-20 23:01:29 +03:00
Konstantin Lopuhin
c6d013ec85 py3 fetch command: it may actually be better to try to print bytes as-is 2016-01-20 23:01:29 +03:00
Konstantin Lopuhin
47d3c63338 py3: port fetch and shell commands, and review + enable already passing test_closespider.py and tests/test_utils_template.py 2016-01-20 23:01:29 +03:00
Konstantin Lopuhin
c1db60188a py3: unskip passing test_engine 2016-01-20 22:57:46 +03:00
Konstantin Lopuhin
0b08b4bfcf fix engine tests - this callback (spider_closed_callback) should accept one argument, but the error was hidden on py2 2016-01-20 22:57:46 +03:00
Konstantin Lopuhin
a32b59ac75 py3: fix EngineTest.test_crawler 2016-01-20 22:57:46 +03:00
Daniel Graña
56b69d2ea8 Merge pull request #1689 from scrapy/remove-lsprof 
[MRG+1] Remove --lsprof command-line option.
 2016-01-20 16:52:25 -03:00
Elias Dorneles
8fb9a6f819 Merge pull request #1674 from redapple/tox-py35 
[MRG+1] Add Python 3.5 tox env + Python 3.5 tests in Travis
 2016-01-20 17:23:25 -02:00
Elias Dorneles
8a1255d3ba Merge pull request #1680 from lopuhin/py3-downloader-middleware 
[MRG+1] Py3: port downloader cache and compression middlewares
 2016-01-20 17:05:54 -02:00
Elias Dorneles
b8497513e7 Merge pull request #1701 from lopuhin/cleanup-http11-tunneling 
[MRG+1] Cleanup http11 tunneling connection after #1678
 2016-01-20 16:38:06 -02:00
Paul Tremberth
29ff84a792 Invert PY2/PY3 test for conditional read1() definition 2016-01-20 12:03:38 +01:00
Konstantin Lopuhin
f0cf5463c8 cleanup http11 tunneling connection after #1678 - construct string and then convert to bytes 2016-01-20 12:23:48 +03:00
Elias Dorneles
ef520541fc Merge pull request #1699 from scrapy/no-optional-features 
[MRG+1] optional_features has been removed
 2016-01-19 16:45:08 -02:00
Daniel Graña
176610f910 optional_features has been removed 2016-01-19 15:34:26 -03:00
Daniel Graña
5ec4319885 Merge pull request #1358 from nyov/nyov/boto-anon-connection 
[MRG+1] Support anonymous S3DownloadHandler (boto) connections
 2016-01-19 13:35:23 -03:00
Daniel Graña
b1e44436bc Merge pull request #1642 from orangain/doc-metarefresh 
[MRG+1] DOC: Update MetaRefreshMiddlware's setting variables
 2016-01-19 13:33:50 -03:00
Daniel Graña
d59d3f1e29 Merge pull request #1694 from lopuhin/py3-twisted-version 
[MRG+1] Raise minimal twisted version for py3
 2016-01-19 13:32:36 -03:00
Elias Dorneles
e8b26e2ab2 Merge pull request #1698 from carlosp420/typo-you 
fixed typo You -> you
 2016-01-19 14:18:43 -02:00
carlosp420
e15f361b05 fixed typo You -> you 2016-01-19 11:12:43 -05:00
Elias Dorneles
b4fb9d3534 Merge pull request #1678 from lopuhin/py3-http-downloaders 
[MRG+1] Py3: port http downloaders
 2016-01-19 14:10:00 -02:00
Paul Tremberth
2b5245839c Remove unused import statement 2016-01-19 17:04:57 +01:00
Paul Tremberth
1f2233837a Use if Py2/Py3 function instead of custom GzipFile class method 2016-01-19 16:58:24 +01:00
Konstantin Lopuhin
4c84172840 move leveldb to tests/requirements-py3.txt 2016-01-19 18:24:58 +03:00
Konstantin Lopuhin
076ef5537e Merge branch 'master' into py3-http-downloaders 2016-01-19 17:51:42 +03:00
Konstantin Lopuhin
07577c432e Merge branch 'master' into py3-http-downloaders 2016-01-19 17:39:22 +03:00
Konstantin Lopuhin
9c3117a914 more pythonic check of noconnect in proxy params 2016-01-19 17:32:53 +03:00
Konstantin Lopuhin
bb50c0be2f remove unused import 2016-01-19 17:30:59 +03:00
Nicolás Alejandro Ramírez Quiros
6827eab2c5 Merge pull request #1654 from aron-bordin/show_maxdownsize_warn_once 
[MRG+1] show download warnsize once
 2016-01-19 11:30:54 -03:00
Konstantin Lopuhin
5ac54ed339 raise minimal twisted version for py3 2016-01-19 17:25:50 +03:00
Daniel Graña
49fe631d89 Merge pull request #1676 from lopuhin/py3-webclient 
[MRG+1] py3: fix webclient
 2016-01-19 11:07:54 -03:00
Mikhail Korobov
fdc3c9d561 Merge pull request #1624 from side2k/more_verbose_download_cancel 
[MRG+1] added more verbosity for log and for exception when download is cancelled because of a size limit
 2016-01-19 00:17:56 +05:00
Mikhail Korobov
ff235fa19a Remove --lsprof command-line option. Fixes GH-1531 2016-01-19 00:00:31 +05:00
Paul Tremberth
fd99ef86df Test for AttributeError when pickling objects (Python>=3.5) 
Same "fix" as in e.g. https://github.com/joblib/joblib/pull/246
 2016-01-18 18:14:33 +01:00
Konstantin Lopuhin
de98d8d006 move comment about test skipped on py3 into a proper place 2016-01-18 19:27:31 +03:00
Konstantin Lopuhin
7ffcc58759 Merge branch 'py3-http-downloaders' into py3-downloader-middleware 2016-01-18 19:23:12 +03:00
Konstantin Lopuhin
7fdd3225b2 fix test skipping logic - this is (temporary) py2-only part 2016-01-18 19:09:09 +03:00
Konstantin Lopuhin
120fb4adeb revert bogus change 2016-01-18 19:00:40 +03:00
Paul Tremberth
bcbad2905d Stick with ValueError for queue/serialization exception tests 2016-01-18 15:22:29 +01:00
Konstantin Lopuhin
324f2c1a6f common test_downloadermiddleware.py also passes now due to fixes in compression middleware 2016-01-18 16:45:22 +03:00
Konstantin Lopuhin
0b9336418e py3: port compression downloader middleware and tests 2016-01-18 16:43:58 +03:00
Konstantin Lopuhin
6edd4dec33 Merge branch 'py3-http-downloaders' into py3-downloader-middleware 2016-01-18 15:51:26 +03:00
Konstantin Lopuhin
f876654a0c Merge branch 'py3-webclient' into py3-http-downloaders 2016-01-18 15:50:58 +03:00
Konstantin Lopuhin
4946434582 revert most changes to this test, and clarify - it is valid only on py2, because urls are strictly unicode on py3 2016-01-18 15:23:01 +03:00
Konstantin Lopuhin
a2efd389b0 clarify: rename r_transform to response_transform 2016-01-18 15:09:54 +03:00
Konstantin Lopuhin
b940606b7e this is a test for TunnelingTCP4ClientEndpoint - move into Http11ProxyTestCase 2016-01-18 15:06:15 +03:00
Elias Dorneles
bc1f5353dd Merge pull request #1671 from redapple/xmliter-unicode 
[MRG+1] Support unicode tags in xml iterators (fixes #1665)
 2016-01-18 10:02:18 -02:00
Konstantin Lopuhin
7af64e8fd2 py3: remove extra encoding/decoding of url: pass it as bytes only when required 2016-01-18 15:00:43 +03:00
Mikhail Korobov
66f41aba3c Merge pull request #1687 from palego/extra-space 
[MRG+1] fix indentation
 2016-01-18 16:48:42 +05:00
Konstantin Lopuhin
0f527849f2 https proxy tunneling - add a test (not perfect, but covers all impl) and fix for py3 2016-01-18 14:44:04 +03:00
Konstantin Lopuhin
98c060d0b2 py3: fix http 1.1 test with https, and use self.host everywhere 2016-01-18 12:42:21 +03:00