BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-26 18:04:27 +00:00
Code Releases Wiki Activity
5,353 Commits 39 Branches 119 Tags
Commit Graph

393 Commits

Author SHA1 Message Date
Paul Tremberth
1a30a7774b Use pytest.mark.parametrize decorator 2016-01-22 18:22:19 +01:00
Paul Tremberth
61ca8d587a Merge remote-tracking branch 'origin/master' into 1550-shell_file-cont 2016-01-22 18:16:36 +01:00
Paul Tremberth
7a51d370f3 Regex-based guess_scheme() + refactor tests 2016-01-22 17:16:27 +01:00
Paul Tremberth
6d73e057b5 Extract guess_scheme function and refactor tests 2016-01-22 13:07:42 +01:00
Mikhail Korobov
35ada10729 PY3 enable tests for scrapy parse command 
scrapy parse command is already ported
 2016-01-22 13:39:27 +05:00
Paul Tremberth
8bd5b60889 Remove relpath filepath 2016-01-21 23:23:50 +01:00
Paul Tremberth
240ecbf323 Add local file tests for scrapy shell command 
Continuation of #1579
 2016-01-21 22:59:48 +01:00
Konstantin Lopuhin
5813de8838 py3: unskip test_closespider - it passes after fixing mockserver.Follow resouce on py3 2016-01-21 17:53:08 +03:00
Konstantin Lopuhin
4607f2843e py3: unskip test_crawl 2016-01-21 17:53:08 +03:00
Konstantin Lopuhin
bf5f54fa33 py3: fix getarg 2016-01-21 17:52:12 +03:00
Konstantin Lopuhin
ad2b3321b9 py3 compat: use range, fixes CrawlTestCase.test_start_requests_bug_yielding 2016-01-21 17:52:12 +03:00
Konstantin Lopuhin
0680950b98 py3: pass CrawlTestCase.test_referer_header, fixing Echo resource in mockserver and json decoding in test 2016-01-21 17:52:12 +03:00
Konstantin Lopuhin
20b839b44b py3: pass first crawl test (test_follow_all): fix mock server 2016-01-21 17:52:12 +03:00
Daniel Graña
0f500a1f82 Merge pull request #1708 from scrapy/fix-scrapy-bench 
PY3 fixed scrapy bench command
 2016-01-21 11:51:27 -03:00
Raul Gallegos
a06a5f00f4 adding configurable encoding for httpproxy authentication 2016-01-21 09:35:04 -05:00
Raul Gallegos
f042ad0f39 py3 fix HttpProxy and Retry Middlewares 2016-01-21 09:35:04 -05:00
Paul Tremberth
c9e046d11d Merge pull request #1657 from palego/os.mknod-fails-on-macos 
[MRG+1] change os.mknod() for open()
 2016-01-21 15:17:57 +01:00
Mikhail Korobov
80c55f19a1 PY3 fixed scrapy bench command 2016-01-21 18:31:58 +05:00
Paul Tremberth
da83f2dca7 Merge pull request #1693 from lopuhin/py3-commands-fetch-shell 
[MRG+1] Py3: port commands fetch and shell
 2016-01-21 10:59:32 +01:00
Paul Tremberth
6d8620ffe7 Merge pull request #1691 from lopuhin/py3-test-engine 
[MRG+1] Py3: port test_engine
 2016-01-21 10:30:39 +01:00
Capi Etheriel
659715ecd9 implements FormRequest.from_response CSS support 2016-01-21 01:05:20 -02:00
Konstantin Lopuhin
a5da7531c4 py3 backout skipping test_closespider - it was fixed on another branch 2016-01-20 23:01:29 +03:00
Konstantin Lopuhin
47d3c63338 py3: port fetch and shell commands, and review + enable already passing test_closespider.py and tests/test_utils_template.py 2016-01-20 23:01:29 +03:00
Konstantin Lopuhin
c1db60188a py3: unskip passing test_engine 2016-01-20 22:57:46 +03:00
Konstantin Lopuhin
0b08b4bfcf fix engine tests - this callback (spider_closed_callback) should accept one argument, but the error was hidden on py2 2016-01-20 22:57:46 +03:00
Konstantin Lopuhin
a32b59ac75 py3: fix EngineTest.test_crawler 2016-01-20 22:57:46 +03:00
Elias Dorneles
8a1255d3ba Merge pull request #1680 from lopuhin/py3-downloader-middleware 
[MRG+1] Py3: port downloader cache and compression middlewares
 2016-01-20 17:05:54 -02:00
Daniel Graña
176610f910 optional_features has been removed 2016-01-19 15:34:26 -03:00
Daniel Graña
5ec4319885 Merge pull request #1358 from nyov/nyov/boto-anon-connection 
[MRG+1] Support anonymous S3DownloadHandler (boto) connections
 2016-01-19 13:35:23 -03:00
Konstantin Lopuhin
4c84172840 move leveldb to tests/requirements-py3.txt 2016-01-19 18:24:58 +03:00
Konstantin Lopuhin
07577c432e Merge branch 'master' into py3-http-downloaders 2016-01-19 17:39:22 +03:00
Daniel Graña
49fe631d89 Merge pull request #1676 from lopuhin/py3-webclient 
[MRG+1] py3: fix webclient
 2016-01-19 11:07:54 -03:00
Konstantin Lopuhin
de98d8d006 move comment about test skipped on py3 into a proper place 2016-01-18 19:27:31 +03:00
Konstantin Lopuhin
7ffcc58759 Merge branch 'py3-http-downloaders' into py3-downloader-middleware 2016-01-18 19:23:12 +03:00
Konstantin Lopuhin
7fdd3225b2 fix test skipping logic - this is (temporary) py2-only part 2016-01-18 19:09:09 +03:00
Konstantin Lopuhin
324f2c1a6f common test_downloadermiddleware.py also passes now due to fixes in compression middleware 2016-01-18 16:45:22 +03:00
Konstantin Lopuhin
0b9336418e py3: port compression downloader middleware and tests 2016-01-18 16:43:58 +03:00
Konstantin Lopuhin
6edd4dec33 Merge branch 'py3-http-downloaders' into py3-downloader-middleware 2016-01-18 15:51:26 +03:00
Konstantin Lopuhin
f876654a0c Merge branch 'py3-webclient' into py3-http-downloaders 2016-01-18 15:50:58 +03:00
Konstantin Lopuhin
4946434582 revert most changes to this test, and clarify - it is valid only on py2, because urls are strictly unicode on py3 2016-01-18 15:23:01 +03:00
Konstantin Lopuhin
a2efd389b0 clarify: rename r_transform to response_transform 2016-01-18 15:09:54 +03:00
Konstantin Lopuhin
b940606b7e this is a test for TunnelingTCP4ClientEndpoint - move into Http11ProxyTestCase 2016-01-18 15:06:15 +03:00
Konstantin Lopuhin
0f527849f2 https proxy tunneling - add a test (not perfect, but covers all impl) and fix for py3 2016-01-18 14:44:04 +03:00
Konstantin Lopuhin
98c060d0b2 py3: fix http 1.1 test with https, and use self.host everywhere 2016-01-18 12:42:21 +03:00
Konstantin Lopuhin
04f69fd184 add https 1.1 downloader test - localhost is a valid DNS-ID 2016-01-18 12:37:46 +03:00
Konstantin Lopuhin
f403c88053 Merge branch 'py3-webclient' into py3-http-downloaders 2016-01-18 12:25:51 +03:00
palego
6b905a9aec split-up the assertIn 
to deal with OS X intricacies (directories prefixed with /private)
 2016-01-18 10:18:31 +01:00
Konstantin Lopuhin
673df5e416 add webclient test - check that non-standart body encoding matches Content-Encoding header 2016-01-18 11:44:49 +03:00
Konstantin Lopuhin
7d44c5dcea py3: unskip tests/test_downloadermiddleware_httpcache.py and scrapy/downloadermiddlewares/httpcache.py 2016-01-15 15:41:31 +03:00
Konstantin Lopuhin
8330776c21 fix error reporting in test: we can fail in process_request too, so result should always be defined 2016-01-15 15:16:12 +03:00