BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-25 09:24:20 +00:00
Code Releases Wiki Activity
5,437 Commits 39 Branches 119 Tags
Commit Graph

5437 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Konstantin Lopuhin
7fdd3225b2 fix test skipping logic - this is (temporary) py2-only part 2016-01-18 19:09:09 +03:00
Konstantin Lopuhin
120fb4adeb revert bogus change 2016-01-18 19:00:40 +03:00
Paul Tremberth
bcbad2905d Stick with ValueError for queue/serialization exception tests 2016-01-18 15:22:29 +01:00
Konstantin Lopuhin
324f2c1a6f common test_downloadermiddleware.py also passes now due to fixes in compression middleware 2016-01-18 16:45:22 +03:00
Konstantin Lopuhin
0b9336418e py3: port compression downloader middleware and tests 2016-01-18 16:43:58 +03:00
Konstantin Lopuhin
6edd4dec33 Merge branch 'py3-http-downloaders' into py3-downloader-middleware 2016-01-18 15:51:26 +03:00
Konstantin Lopuhin
f876654a0c Merge branch 'py3-webclient' into py3-http-downloaders 2016-01-18 15:50:58 +03:00
Konstantin Lopuhin
4946434582 revert most changes to this test, and clarify - it is valid only on py2, because urls are strictly unicode on py3 2016-01-18 15:23:01 +03:00
Konstantin Lopuhin
a2efd389b0 clarify: rename r_transform to response_transform 2016-01-18 15:09:54 +03:00
Konstantin Lopuhin
b940606b7e this is a test for TunnelingTCP4ClientEndpoint - move into Http11ProxyTestCase 2016-01-18 15:06:15 +03:00
Elias Dorneles
bc1f5353dd Merge pull request #1671 from redapple/xmliter-unicode 
[MRG+1] Support unicode tags in xml iterators (fixes #1665)
 2016-01-18 10:02:18 -02:00
Konstantin Lopuhin
7af64e8fd2 py3: remove extra encoding/decoding of url: pass it as bytes only when required 2016-01-18 15:00:43 +03:00
Mikhail Korobov
66f41aba3c Merge pull request #1687 from palego/extra-space 
[MRG+1] fix indentation
 2016-01-18 16:48:42 +05:00
Konstantin Lopuhin
0f527849f2 https proxy tunneling - add a test (not perfect, but covers all impl) and fix for py3 2016-01-18 14:44:04 +03:00
Konstantin Lopuhin
98c060d0b2 py3: fix http 1.1 test with https, and use self.host everywhere 2016-01-18 12:42:21 +03:00
Konstantin Lopuhin
04f69fd184 add https 1.1 downloader test - localhost is a valid DNS-ID 2016-01-18 12:37:46 +03:00
Konstantin Lopuhin
f403c88053 Merge branch 'py3-webclient' into py3-http-downloaders 2016-01-18 12:25:51 +03:00
Konstantin Lopuhin
f2e2ff5e1f py3: webclient assumes that urls come from Request.url and are ascii-only 2016-01-18 12:20:01 +03:00
palego
6b905a9aec split-up the assertIn 
to deal with OS X intricacies (directories prefixed with /private)
 2016-01-18 10:18:31 +01:00
Konstantin Lopuhin
673df5e416 add webclient test - check that non-standart body encoding matches Content-Encoding header 2016-01-18 11:44:49 +03:00
Konstantin Lopuhin
d3c2b0cf7e py3 webclient: I was mistaken about unicode body, revert conversion to bytes and fix HEAD response 2016-01-18 10:06:14 +03:00
Mikhail Korobov
9ef25d7b68 Merge pull request #1686 from cclauss/patch-1 
Simplify if statement
 2016-01-18 11:57:06 +05:00
cclauss
cd735e377c Simplify if statement 2016-01-18 07:45:36 +01:00
Paul Tremberth
4e44766653 Use "unicode" string for lxml.etree.tostring() serialization 2016-01-15 19:51:21 +01:00
Elias Dorneles
3f1f15bc4d Merge pull request #1683 from rgtk/patch-2 
Update Stats Collection documentation for @master
 2016-01-15 16:30:52 -02:00
Ralph Gutkowski
79147a61a7 Update stats.rst 2016-01-15 19:25:56 +01:00
Ralph Gutkowski
bb38400db5 Update Stats Collection documentation 
`pages_crawled` value doesn't exist. Replace it with `downloader/response_count` in order to avoid confusion.
 2016-01-15 19:00:58 +01:00
Paul Tremberth
73d78ec99f Add Code of Conduct Version 1.3.0 from http://contributor-covenant.org/ 
Closes #1645
 2016-01-15 18:01:04 +01:00
Paul Tremberth
ee4fadc007 Use .read1() if available when using GzipFile 2016-01-15 14:57:15 +01:00
Konstantin Lopuhin
7d44c5dcea py3: unskip tests/test_downloadermiddleware_httpcache.py and scrapy/downloadermiddlewares/httpcache.py 2016-01-15 15:41:31 +03:00
Konstantin Lopuhin
085fdd6283 py3 fix for ignoring cache controls - map is not a list 2016-01-15 15:40:45 +03:00
Konstantin Lopuhin
b0648271d6 py3 fix for rfc1123_to_epoch - "except Exception" was hiding bytes/str error 2016-01-15 15:17:30 +03:00
Konstantin Lopuhin
8330776c21 fix error reporting in test: we can fail in process_request too, so result should always be defined 2016-01-15 15:16:12 +03:00
Konstantin Lopuhin
4398d95a02 skip this file on py3 again - it has one compression test, sould be done separately 2016-01-15 14:54:12 +03:00
Konstantin Lopuhin
ae6758ee97 Merge branch 'py3-http-downloaders' into py3-downloader-middleware 2016-01-15 14:39:27 +03:00
Konstantin Lopuhin
a4ca1668d8 add https test for http10 handler (no luck with testing https with http11 so far) 2016-01-15 14:20:19 +03:00
Konstantin Lopuhin
96fcf4cea4 add a check that byte url is not accepted in http.Response on py3 2016-01-15 13:27:28 +03:00
Paul Tremberth
131f463247 Allow failures for Python 3.5 for now 2016-01-15 11:17:52 +01:00
Konstantin Lopuhin
87849780bc some py3 fixes for RFC2616Policy 2016-01-15 13:13:10 +03:00
Konstantin Lopuhin
ea0471e33a py3: fix LeveldbCacheStorage - using bytes as keys and values in leveldb 2016-01-15 12:22:41 +03:00
Konstantin Lopuhin
ddc91dda27 py3: fix _BaseTest in httpcache 2016-01-15 11:49:28 +03:00
Konstantin Lopuhin
e7ed1fd70d py3 compat in httpcache - headers are bytes 2016-01-15 11:48:07 +03:00
Konstantin Lopuhin
dbf6cc73d9 py3: add leveldb to py33 test env, fix anydbm module name on py3 2016-01-15 11:46:56 +03:00
Konstantin Lopuhin
94ab7bee6c py3: body to bytes in tests, unskip test file 2016-01-15 11:26:01 +03:00
Konstantin Lopuhin
99f1f2ad1d unskip tests and modules ported to py3 2016-01-14 19:00:48 +03:00
Konstantin Lopuhin
81a90c3af6 unskip part of test_download_gzip_response on py3, file a twisted issue for the remaining part 2016-01-14 18:47:06 +03:00
Konstantin Lopuhin
2aa6c92ffc py3 fixes in tests.mockserver 2016-01-14 17:52:50 +03:00
Konstantin Lopuhin
f46a9d595d skip ftp tests on py3 for now 2016-01-14 17:31:58 +03:00
Konstantin Lopuhin
4950f5988e py3: pass http proxy tests 2016-01-14 17:24:08 +03:00
Konstantin Lopuhin
c6f14a39de py3: fix http10 downloader - unicode host expected here 2016-01-14 16:50:16 +03:00