BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-24 23:43:59 +00:00
Code Releases Wiki Activity
5,344 Commits 39 Branches 119 Tags
Commit Graph

5344 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Konstantin Lopuhin
097082cffa reviewed py3 compat in pipelines/images.py and pipelines/files.py 2016-01-24 23:05:23 +03:00
Konstantin Lopuhin
333d4c91fb py3: add boto to py3 test requirements, test_pipeline_files and test_pipeline_images passing now 2016-01-24 22:55:21 +03:00
Mikhail Korobov
85ae273230 Merge pull request #1715 from lopuhin/py3-test-mail 
[MRG+1] py3: fix test_mail
 2016-01-24 17:47:57 +05:00
Konstantin Lopuhin
860353b0c0 py3: unskip test_mail and scrapy/mail.py 2016-01-24 13:27:41 +03:00
Konstantin Lopuhin
9704226ee4 py3: fix test_mail - get_payload returns bytes when decode is True 2016-01-24 13:25:14 +03:00
Mikhail Korobov
c75f7ed20b Merge pull request #1711 from scrapy/py3-parse-command-tests 
PY3 enable tests for scrapy parse command
 2016-01-22 15:21:26 +05:00
Mikhail Korobov
35ada10729 PY3 enable tests for scrapy parse command 
scrapy parse command is already ported
 2016-01-22 13:39:27 +05:00
Elias Dorneles
e0074d7f4d Merge pull request #1692 from lopuhin/py3-test-crawl 
[MRG+1] Py3: port test crawl
 2016-01-21 13:05:20 -02:00
Konstantin Lopuhin
5813de8838 py3: unskip test_closespider - it passes after fixing mockserver.Follow resouce on py3 2016-01-21 17:53:08 +03:00
Konstantin Lopuhin
4607f2843e py3: unskip test_crawl 2016-01-21 17:53:08 +03:00
Konstantin Lopuhin
bf5f54fa33 py3: fix getarg 2016-01-21 17:52:12 +03:00
Konstantin Lopuhin
ad2b3321b9 py3 compat: use range, fixes CrawlTestCase.test_start_requests_bug_yielding 2016-01-21 17:52:12 +03:00
Konstantin Lopuhin
0680950b98 py3: pass CrawlTestCase.test_referer_header, fixing Echo resource in mockserver and json decoding in test 2016-01-21 17:52:12 +03:00
Konstantin Lopuhin
20b839b44b py3: pass first crawl test (test_follow_all): fix mock server 2016-01-21 17:52:12 +03:00
Daniel Graña
0f500a1f82 Merge pull request #1708 from scrapy/fix-scrapy-bench 
PY3 fixed scrapy bench command
 2016-01-21 11:51:27 -03:00
Daniel Graña
5346011eaa Merge pull request #1637 from eLRuLL/py3-httpproxy-retry 
[MRG+1] py3 fix HttpProxy and Retry Middlewares
 2016-01-21 11:49:42 -03:00
Raul Gallegos
a06a5f00f4 adding configurable encoding for httpproxy authentication 2016-01-21 09:35:04 -05:00
Raul Gallegos
f042ad0f39 py3 fix HttpProxy and Retry Middlewares 2016-01-21 09:35:04 -05:00
Paul Tremberth
c9e046d11d Merge pull request #1657 from palego/os.mknod-fails-on-macos 
[MRG+1] change os.mknod() for open()
 2016-01-21 15:17:57 +01:00
Mikhail Korobov
a18dc24471 correctly process arguments for bench server 2016-01-21 18:44:37 +05:00
Mikhail Korobov
80c55f19a1 PY3 fixed scrapy bench command 2016-01-21 18:31:58 +05:00
Mikhail Korobov
e44ce8634f Merge pull request #1706 from scrapy/fix-form-request-formcss-version 
Fix version number to appear new feature
 2016-01-21 16:48:21 +05:00
Elias Dorneles
d4c4ca8062 fix version number to appear new feature 2016-01-21 09:42:15 -02:00
Paul Tremberth
da83f2dca7 Merge pull request #1693 from lopuhin/py3-commands-fetch-shell 
[MRG+1] Py3: port commands fetch and shell
 2016-01-21 10:59:32 +01:00
Paul Tremberth
6d8620ffe7 Merge pull request #1691 from lopuhin/py3-test-engine 
[MRG+1] Py3: port test_engine
 2016-01-21 10:30:39 +01:00
Elias Dorneles
a6e5c848fe Merge pull request #1382 from barraponto/formcss 
[MRG+1] Implement FormRequest.from_response CSS support
 2016-01-21 01:17:47 -02:00
Capi Etheriel
659715ecd9 implements FormRequest.from_response CSS support 2016-01-21 01:05:20 -02:00
Konstantin Lopuhin
fd24e22442 use byte constants for prefix instead of encoding it 2016-01-20 23:30:58 +03:00
Konstantin Lopuhin
a5da7531c4 py3 backout skipping test_closespider - it was fixed on another branch 2016-01-20 23:01:29 +03:00
Konstantin Lopuhin
c6d013ec85 py3 fetch command: it may actually be better to try to print bytes as-is 2016-01-20 23:01:29 +03:00
Konstantin Lopuhin
47d3c63338 py3: port fetch and shell commands, and review + enable already passing test_closespider.py and tests/test_utils_template.py 2016-01-20 23:01:29 +03:00
Konstantin Lopuhin
c1db60188a py3: unskip passing test_engine 2016-01-20 22:57:46 +03:00
Konstantin Lopuhin
0b08b4bfcf fix engine tests - this callback (spider_closed_callback) should accept one argument, but the error was hidden on py2 2016-01-20 22:57:46 +03:00
Konstantin Lopuhin
a32b59ac75 py3: fix EngineTest.test_crawler 2016-01-20 22:57:46 +03:00
Daniel Graña
56b69d2ea8 Merge pull request #1689 from scrapy/remove-lsprof 
[MRG+1] Remove --lsprof command-line option.
 2016-01-20 16:52:25 -03:00
Elias Dorneles
8fb9a6f819 Merge pull request #1674 from redapple/tox-py35 
[MRG+1] Add Python 3.5 tox env + Python 3.5 tests in Travis
 2016-01-20 17:23:25 -02:00
Elias Dorneles
8a1255d3ba Merge pull request #1680 from lopuhin/py3-downloader-middleware 
[MRG+1] Py3: port downloader cache and compression middlewares
 2016-01-20 17:05:54 -02:00
Elias Dorneles
b8497513e7 Merge pull request #1701 from lopuhin/cleanup-http11-tunneling 
[MRG+1] Cleanup http11 tunneling connection after #1678
 2016-01-20 16:38:06 -02:00
Paul Tremberth
29ff84a792 Invert PY2/PY3 test for conditional read1() definition 2016-01-20 12:03:38 +01:00
Konstantin Lopuhin
f0cf5463c8 cleanup http11 tunneling connection after #1678 - construct string and then convert to bytes 2016-01-20 12:23:48 +03:00
Elias Dorneles
ef520541fc Merge pull request #1699 from scrapy/no-optional-features 
[MRG+1] optional_features has been removed
 2016-01-19 16:45:08 -02:00
Daniel Graña
176610f910 optional_features has been removed 2016-01-19 15:34:26 -03:00
Daniel Graña
5ec4319885 Merge pull request #1358 from nyov/nyov/boto-anon-connection 
[MRG+1] Support anonymous S3DownloadHandler (boto) connections
 2016-01-19 13:35:23 -03:00
Daniel Graña
b1e44436bc Merge pull request #1642 from orangain/doc-metarefresh 
[MRG+1] DOC: Update MetaRefreshMiddlware's setting variables
 2016-01-19 13:33:50 -03:00
Daniel Graña
d59d3f1e29 Merge pull request #1694 from lopuhin/py3-twisted-version 
[MRG+1] Raise minimal twisted version for py3
 2016-01-19 13:32:36 -03:00
Elias Dorneles
e8b26e2ab2 Merge pull request #1698 from carlosp420/typo-you 
fixed typo You -> you
 2016-01-19 14:18:43 -02:00
carlosp420
e15f361b05 fixed typo You -> you 2016-01-19 11:12:43 -05:00
Elias Dorneles
b4fb9d3534 Merge pull request #1678 from lopuhin/py3-http-downloaders 
[MRG+1] Py3: port http downloaders
 2016-01-19 14:10:00 -02:00
Paul Tremberth
2b5245839c Remove unused import statement 2016-01-19 17:04:57 +01:00
Paul Tremberth
1f2233837a Use if Py2/Py3 function instead of custom GzipFile class method 2016-01-19 16:58:24 +01:00