BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-27 07:24:32 +00:00
Code Releases Wiki Activity
5,487 Commits 39 Branches 119 Tags
Commit Graph

5487 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Konstantin Lopuhin
408bc1580b Pass env variables required for running tests against real s3 via tox. 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin
bcb92b50dc check that no extra kwargs are silently discarded 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin
d6bea3bf2e botocore not only does not allow passing our own Date header, but does not handle x-amz-date according to the spec 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin
7748ee6bba mock date in s3 tests when using botocore 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin
c3fec83e7e use botocore by default, boto is still used in "precise" env 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin
1b1092b7d0 add Headers.to_native_string_dict - useful when interfacing with other libraries 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin
467553cc29 fix anon test: in this case we do no signing, just change the url 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin
eaf3a239e4 using botocore for s3 request signing: proof of concept 2016-02-15 19:59:46 +03:00
Paul Tremberth
41588397c0 Merge pull request #1765 from scrapy/add-deprecation-for-pydispatch 
[MRG+1] Add fallback and deprecation warning for pydispatch (fixes #1762)
 2016-02-11 19:29:53 +01:00
Elias Dorneles
164493df2e add deprecation for pydispatch (thanks for the help @redapple) 2016-02-11 16:15:28 -02:00
Elias Dorneles
150d1c370b Merge pull request #1769 from orangain/stdout-typeerror 
[MRG+1] PY3: Fix TypeError when outputting to stdout
 2016-02-10 14:30:11 -02:00
Paul Tremberth
c083935806 Merge pull request #1771 from orangain/secure-cookies 
[MRG+1] PY3: Implement some attributes of WrappedRequest required in Python 3
 2016-02-08 11:52:00 +01:00
Mikhail Korobov
44bc4c06ff Merge pull request #1767 from orangain/sitemap-robotstxt 
[MRG+1] PY3: Fix SitemapSpider to extract sitemap urls from robots.txt properly
 2016-02-08 10:11:58 +05:00
orangain
1f743996ff PY3: Implement some attributes of WrappedRequest required in Python 3 
This will fix #1770.
 2016-02-07 14:19:27 +09:00
orangain
1cb841bfb3 PY3: Fix TypeError when outputting to stdout 
This will fix #1768.
 2016-02-07 10:06:42 +09:00
orangain
25c56159b8 Fix SitemapSpider to extract sitemap urls from robots.txt properly 
This will fix #1766.
 2016-02-06 23:54:07 +09:00
Elias Dorneles
e328a9b9df Merge pull request #1750 from scrapy/spider-is-idle 
[MRG+1] more efficient ExecutionEngine.spider_is_idle
 2016-02-05 21:18:35 -02:00
Elias Dorneles
daf0f660d1 Merge pull request #1662 from NicolasP/send_utf8 
[MRG+1] MailSender.send: allow passing a charset.
 2016-02-04 17:18:19 -02:00
Nicolas Pennequin
6efc7a9de0 Update the email doc for the charset argument to send(). 2016-02-04 19:33:44 +01:00
Nicolas Pennequin
061c63592a MailSender.send: allow passing a charset. 
Resolves Issue #348
 2016-02-04 19:33:44 +01:00
Daniel Graña
8f269558f1 Bump version: 1.2.0dev1 → 1.2.0dev2 1.2.0dev2 2016-02-03 22:58:21 -03:00
Daniel Graña
6cf4fdbae6 Enable travis builds on tag patterns 2016-02-03 22:56:49 -03:00
Elias Dorneles
aaccdbb49b Bump version: 1.1.0rc1 → 1.2.0dev1 2016-02-03 21:54:39 -02:00
Elias Dorneles
a0b63a7707 Bump version: 1.1.0dev1 → 1.1.0rc1 2016-02-03 21:52:47 -02:00
Elias Dorneles
94b2c47da5 Merge pull request #1751 from scrapy/release-notes-1.1 
[MRG+1] Release notes 1.1
 2016-02-03 21:51:10 -02:00
Elias Dorneles
ca08314654 makes more explicit source of problems in Python 3 limitations 2016-02-03 21:38:25 -02:00
Elias Dorneles
2167573b32 fix comment about disabling robots.txt 2016-02-03 21:31:27 -02:00
Elias Dorneles
3180abc800 applying more review comments 2016-02-03 21:28:05 -02:00
Elias Dorneles
d40add7b27 add note about robots.txt waiting and make it explicit builtin extensions only are ported 2016-02-03 21:18:25 -02:00
Elias Dorneles
9cfefd5277 favor beta over basic to describe PY3 support 2016-02-03 20:59:56 -02:00
Elias Dorneles
2f697ce59f added note about how to revert to old behavior for robotstxt 2016-02-03 20:51:26 -02:00
Elias Dorneles
89a088b3c3 applying review comments 2016-02-03 20:45:55 -02:00
Daniel Graña
a214b131fc Merge pull request #1748 from scrapy/py3-crawlerrunner-stop 
[MRG+1] PY3 fixed CrawlerRunner.stop
 2016-02-03 19:41:43 -03:00
Paul Tremberth
972f192568 Blank line before CoC 2016-02-03 22:58:49 +01:00
Paul Tremberth
728e505e53 Merge remote-tracking branch 'origin/master' into release-notes-1.1 2016-02-03 22:47:15 +01:00
Paul Tremberth
725a209e83 Merge pull request #1755 from stummjr/add-overview-to-release-notes 
Add overview to release notes and other edits
 2016-02-03 22:37:02 +01:00
Valdir Stumm Jr
dd7b0eb359 Updates to release notes 2016-02-03 19:23:02 -02:00
Paul Tremberth
51dc741fa2 Add 1.0.5 release notes 2016-02-03 21:07:21 +01:00
Valdir Stumm Jr
0f816524e8 edit API changes, Deprecations, and Relocations sections. 2016-02-03 16:21:32 -02:00
Paul Tremberth
fd3193da86 Reword some of "New Features and Enhancements" 2016-02-03 18:19:38 +01:00
Paul Tremberth
3b2a6abc0b Format "Code refactoring" subsection 2016-02-03 17:37:58 +01:00
Paul Tremberth
ed02455010 Format "Logging" subsection 2016-02-03 17:27:43 +01:00
Elias Dorneles
a8a6f050e7 Merge pull request #1735 from ArturGaspar/master 
[MRG+1] Fix for KeyError in robots.txt middleware
 2016-02-03 13:30:06 -02:00
Paul Tremberth
e4cd95444c Merge pull request #1752 from stummjr/release-notes-1.1 
Release notes: edits on "Py3 Support" and "New features" sections
 2016-02-03 16:17:59 +01:00
Mikhail Korobov
43a53aca12 Merge pull request #1746 from redapple/shell-settings-logging 
[MRG+1] Remove __str__ and __repr__ from settings, introduce copy_to_dict()
 2016-02-03 19:59:24 +05:00
Valdir Stumm Jr
1fd95c9c9f fixed references to issues and fit text to 80 cols in the first two sections 2016-02-03 12:45:48 -02:00
Valdir Stumm Jr
a21c90bee7 edits on Py3 Support and New features sections 2016-02-03 11:54:46 -02:00
Paul Tremberth
44d8df2060 Add versionadded note for MEMUSAGE_CHECK_INTERVAL_SECONDS 2016-02-03 12:35:26 +01:00
Paul Tremberth
2b033eebce Fix recently added HTTPCACHE_ settings versionadded notes 2016-02-03 12:34:44 +01:00
Paul Tremberth
142aa21737 Add AUTOTHROTTLE_TARGET_CONCURRENCY versionadded note 2016-02-03 12:33:10 +01:00