BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-26 08:03:54 +00:00
Code Releases Wiki Activity
5,223 Commits 39 Branches 119 Tags
Commit Graph

19 Commits

Author SHA1 Message Date
Konstantin Lopuhin
81a90c3af6 unskip part of test_download_gzip_response on py3, file a twisted issue for the remaining part 2016-01-14 18:47:06 +03:00
Konstantin Lopuhin
f46a9d595d skip ftp tests on py3 for now 2016-01-14 17:31:58 +03:00
Konstantin Lopuhin
4950f5988e py3: pass http proxy tests 2016-01-14 17:24:08 +03:00
Konstantin Lopuhin
c6f14a39de py3: fix http10 downloader - unicode host expected here 2016-01-14 16:50:16 +03:00
Konstantin Lopuhin
6b79fffa9a py3: pass all of HttpTestCase 2016-01-14 16:37:18 +03:00
Konstantin Lopuhin
3509378b8b py3: pass first http downloader test, simple crawler works now, yay! 2016-01-14 16:29:19 +03:00
Konstantin Lopuhin
32bb5b682a fix import of test_downloader_handlers.py: use @implementer, move failing on py3 imports into corresponding tests 2016-01-14 16:11:16 +03:00
nyov
ecbfe4bd66 drop deprecated "optional_features" set 2015-09-15 09:00:08 +00:00
nyov
8c7997083f lazy-loading for DownloadHandlers 2015-08-10 16:52:44 -03:00
nyov
b2fd9bdb63 lazy-load s3 boto 2015-08-01 03:43:03 +00:00
Daniel Graña
ebe889a663 Unquote request path before passing to FTPClient, it already escape paths 2015-05-23 20:50:30 -03:00
Julia Medina
d3f576a816 Move scrapy/spider.py to scrapy/spiders/__init__.py 2015-05-09 04:20:09 -03:00
immerrr
82b187f283 S3DownloadHandler: fix auth for requests with quoted paths/query params 2014-12-11 18:14:36 +03:00
tpeng
cd19382754 attemp to fix travis fails 2014-11-25 14:20:25 +01:00
tpeng
a69f042d10 add 2 more test cases and minor doc fixes 2014-11-19 15:31:07 +01:00
tpeng
fa84730e70 avoid download large response 
introduce DOWNLOAD_MAXSIZE and DOWNLOAD_WARNSIZE in settings and
download_maxsize/download_warnsize in spider/request meta, so
downloader stop downloading as soon as the received data exceed the
limit. also check the twsisted response's length in advance to stop
downloading as early as possible.
 2014-11-12 12:28:02 +01:00
Julia Medina
870438e5f4 Update tests utils, fixing get_crawler and removing docrawl 2014-08-12 14:02:56 -03:00
Daniel Graña
521303e27f fix ftp test cases 2014-08-01 02:10:57 -03:00
Daniel Graña
242c08553f move tests under root dir 2014-07-30 19:45:08 -03:00