BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-25 10:24:14 +00:00
Code Releases Wiki Activity
5,278 Commits 39 Branches 119 Tags
Commit Graph

352 Commits

Author SHA1 Message Date
Daniel Graña
176610f910 optional_features has been removed 2016-01-19 15:34:26 -03:00
Daniel Graña
5ec4319885 Merge pull request #1358 from nyov/nyov/boto-anon-connection 
[MRG+1] Support anonymous S3DownloadHandler (boto) connections
 2016-01-19 13:35:23 -03:00
Konstantin Lopuhin
07577c432e Merge branch 'master' into py3-http-downloaders 2016-01-19 17:39:22 +03:00
Daniel Graña
49fe631d89 Merge pull request #1676 from lopuhin/py3-webclient 
[MRG+1] py3: fix webclient
 2016-01-19 11:07:54 -03:00
Konstantin Lopuhin
de98d8d006 move comment about test skipped on py3 into a proper place 2016-01-18 19:27:31 +03:00
Konstantin Lopuhin
7fdd3225b2 fix test skipping logic - this is (temporary) py2-only part 2016-01-18 19:09:09 +03:00
Konstantin Lopuhin
f876654a0c Merge branch 'py3-webclient' into py3-http-downloaders 2016-01-18 15:50:58 +03:00
Konstantin Lopuhin
4946434582 revert most changes to this test, and clarify - it is valid only on py2, because urls are strictly unicode on py3 2016-01-18 15:23:01 +03:00
Konstantin Lopuhin
a2efd389b0 clarify: rename r_transform to response_transform 2016-01-18 15:09:54 +03:00
Konstantin Lopuhin
b940606b7e this is a test for TunnelingTCP4ClientEndpoint - move into Http11ProxyTestCase 2016-01-18 15:06:15 +03:00
Konstantin Lopuhin
0f527849f2 https proxy tunneling - add a test (not perfect, but covers all impl) and fix for py3 2016-01-18 14:44:04 +03:00
Konstantin Lopuhin
98c060d0b2 py3: fix http 1.1 test with https, and use self.host everywhere 2016-01-18 12:42:21 +03:00
Konstantin Lopuhin
04f69fd184 add https 1.1 downloader test - localhost is a valid DNS-ID 2016-01-18 12:37:46 +03:00
Konstantin Lopuhin
f403c88053 Merge branch 'py3-webclient' into py3-http-downloaders 2016-01-18 12:25:51 +03:00
Konstantin Lopuhin
673df5e416 add webclient test - check that non-standart body encoding matches Content-Encoding header 2016-01-18 11:44:49 +03:00
Konstantin Lopuhin
a4ca1668d8 add https test for http10 handler (no luck with testing https with http11 so far) 2016-01-15 14:20:19 +03:00
Konstantin Lopuhin
96fcf4cea4 add a check that byte url is not accepted in http.Response on py3 2016-01-15 13:27:28 +03:00
Konstantin Lopuhin
99f1f2ad1d unskip tests and modules ported to py3 2016-01-14 19:00:48 +03:00
Konstantin Lopuhin
81a90c3af6 unskip part of test_download_gzip_response on py3, file a twisted issue for the remaining part 2016-01-14 18:47:06 +03:00
Konstantin Lopuhin
2aa6c92ffc py3 fixes in tests.mockserver 2016-01-14 17:52:50 +03:00
Konstantin Lopuhin
f46a9d595d skip ftp tests on py3 for now 2016-01-14 17:31:58 +03:00
Konstantin Lopuhin
4950f5988e py3: pass http proxy tests 2016-01-14 17:24:08 +03:00
Konstantin Lopuhin
c6f14a39de py3: fix http10 downloader - unicode host expected here 2016-01-14 16:50:16 +03:00
Konstantin Lopuhin
6b79fffa9a py3: pass all of HttpTestCase 2016-01-14 16:37:18 +03:00
Konstantin Lopuhin
3509378b8b py3: pass first http downloader test, simple crawler works now, yay! 2016-01-14 16:29:19 +03:00
Konstantin Lopuhin
32bb5b682a fix import of test_downloader_handlers.py: use @implementer, move failing on py3 imports into corresponding tests 2016-01-14 16:11:16 +03:00
Konstantin Lopuhin
5c2241ccc7 py3: fix webclient tests after making ScrapyHTTPClientFactory use bytes as in twisted 2016-01-14 15:30:28 +03:00
Konstantin Lopuhin
6a412d2503 all tests pass in test_webclient.py on py3 - removing from py3-ignores 2016-01-14 13:48:48 +03:00
Konstantin Lopuhin
0178356178 py3 fix testRedirect: url is bytes here 2016-01-14 13:46:46 +03:00
Konstantin Lopuhin
30c7b4e4cc py3 compat in test_timeoutTriggering cleanup 2016-01-14 13:44:14 +03:00
Konstantin Lopuhin
88f55312af py3 fix in testFactoryInfo - factory attirbutes are bytes in twisted 2016-01-14 13:43:37 +03:00
Konstantin Lopuhin
b5f9bc8499 py3 test fixes in test_webclient - expect bytes as page body 2016-01-14 13:04:45 +03:00
Konstantin Lopuhin
ae4aa2c3b2 py3 test fix: putChild expects bytes as path 2016-01-14 13:04:10 +03:00
Konstantin Lopuhin
325b6af6c2 fix ScrapyHTTPPageGetterTests for py3 - we expect bytes here 2016-01-14 12:29:59 +03:00
Konstantin Lopuhin
945674eb8f pass test_externalUnicodeInterference - the logic for py3 is clearly inverse of what was expected in this test, as scrapy Request url must be unicode 2016-01-14 12:25:54 +03:00
Konstantin Lopuhin
1d5ab67183 pass test_getPageHead on py3 2016-01-14 12:04:26 +03:00
Konstantin Lopuhin
9f2be23a39 webclient tests, py3: fix setUp, pass test_getPage 2016-01-14 11:42:23 +03:00
Paul Tremberth
1347015a80 Refactored test code 2016-01-13 12:32:28 +01:00
Paul Tremberth
d4c7d72b2b Add tests for input type in xmliter calls 2016-01-13 12:13:47 +01:00
Paul Tremberth
9fad25f3d1 Use explicit Unicode and bytes for XML body in tests 2016-01-13 11:42:41 +01:00
Paul Tremberth
2f2c2e8096 Merge remote-tracking branch 'origin/master' into xmliter-unicode 2016-01-13 11:42:01 +01:00
Paul Tremberth
6ddd814738 Support unicode tags in xml iterators (fixes #1665) 2016-01-12 10:48:45 +01:00
Paul Tremberth
d4872940db PY3: port utils/iterators 2016-01-07 10:51:27 +01:00
Elias Dorneles
7d187735ff Merge pull request #1359 from nyov/nyov/drop-optional_features 
[MRG+1] drop deprecated "optional_features" set
 2015-12-04 14:08:44 -02:00
Daniel Graña
17aba44f16 Merge pull request #1565 from starrify/fix-1564 
[MRG+1] fixed: Issue #1564 (Incorrectly picked URL in `scrapy.linkextractors.regex.RegexLinkExtractor` when there is a `<base>` tag. )
 2015-12-04 10:31:13 -03:00
Daniel Graña
dc65026395 Merge pull request #1597 from Digenis/various_formreq_fixes 
[MRG+1] Various FormRequest tests+fixes
 2015-12-04 09:55:12 -03:00
palego
2c251000d3 custom project templates 
allow override of TEMPLATES_DIR for startproject
copy full TEMPLATES_DIR/project tree
doc update
 2015-12-01 03:07:27 +01:00
Νικόλαος-Διγενής Καραγιάννης
395ef805eb FormRequest: test unicode xpath expr & exception 2015-11-13 18:54:02 +02:00
Νικόλαος-Διγενής Καραγιάννης
7a438a51b7 FormRequest: test default <input> type (is text) 2015-11-13 18:03:54 +02:00
Νικόλαος-Διγενής Καραγιάννης
881bf19e68 FormRequest: test case-insensitive type attribute 2015-11-13 17:57:25 +02:00