Paul Tremberth
|
da36b7d386
|
Merge pull request #1761 from lopuhin/py3-s3-botocore
[MRG+1] Py3 S3 botocore
|
2016-02-18 15:11:03 +01:00 |
|
Paul Tremberth
|
104027d78d
|
Minor change on quotes
Trying to force Travis CI to build
|
2016-02-18 11:45:03 +01:00 |
|
Konstantin Lopuhin
|
d61fbcc8b5
|
Support headers in S3FilesStore.persist_file for botocore
|
2016-02-18 10:57:02 +03:00 |
|
Konstantin Lopuhin
|
617631f264
|
Fix method name: this always returns unicode keys and values
|
2016-02-18 10:10:16 +03:00 |
|
Mikhail Korobov
|
f766dd0ba8
|
Preserve tracebacks better. Fixes GH-1760.
|
2016-02-17 23:07:03 +05:00 |
|
Mikhail Korobov
|
06da7af9e2
|
TST clean up RunSpiderCommandTest
|
2016-02-17 23:03:12 +05:00 |
|
Mikhail Korobov
|
4e2b7207ec
|
Merge pull request #1786 from redapple/header-encoding
More liberal Content-Disposition header parsing
|
2016-02-17 21:33:43 +05:00 |
|
Paul Tremberth
|
cabed6f183
|
More liberal Content-Disposition header parsing
Fixes #1782
|
2016-02-17 16:55:28 +01:00 |
|
Konstantin Lopuhin
|
49313a6988
|
use absolute_import to import external boto package
|
2016-02-15 20:16:40 +03:00 |
|
Konstantin Lopuhin
|
d1ecb8cd38
|
Fix S3TestCase for precise env: we reraise TypeError as NotConfigured in this case
|
2016-02-15 19:59:48 +03:00 |
|
Konstantin Lopuhin
|
e7c4806c5e
|
Update feedstorage docs: add botocore, mention that boto is supported only on Python 2
|
2016-02-15 19:59:48 +03:00 |
|
Konstantin Lopuhin
|
77ebb13684
|
fix assertRaises for precise env
|
2016-02-15 19:59:48 +03:00 |
|
Konstantin Lopuhin
|
08bc41cc68
|
py3: reviewed s3 downloader handlers
|
2016-02-15 19:59:48 +03:00 |
|
Konstantin Lopuhin
|
3cb7a567ea
|
py3 fix for TestS3FilesStore: checksum is a native string
|
2016-02-15 19:59:48 +03:00 |
|
Konstantin Lopuhin
|
cfc567f48e
|
botocore support for S3FilesStore
|
2016-02-15 19:59:48 +03:00 |
|
Konstantin Lopuhin
|
32cd8c9165
|
add direct test for S3FilesStore
|
2016-02-15 19:59:48 +03:00 |
|
Konstantin Lopuhin
|
d1470e85a2
|
S3FeedStorageTest: pass on py3, add some non-ascii content to be sure
|
2016-02-15 19:59:48 +03:00 |
|
Konstantin Lopuhin
|
3ada45a9bb
|
S3FeedStorageTest: add botocore support, and organize boto/botocore checks
|
2016-02-15 19:59:47 +03:00 |
|
Konstantin Lopuhin
|
5d2f067458
|
S3FeedStorageTest: delete key after test
|
2016-02-15 19:59:47 +03:00 |
|
Konstantin Lopuhin
|
19b2910ad1
|
Fix assert_aws_environ: check for botocore with boto fallback on PY2
|
2016-02-15 19:59:47 +03:00 |
|
Konstantin Lopuhin
|
408bc1580b
|
Pass env variables required for running tests against real s3 via tox.
|
2016-02-15 19:59:47 +03:00 |
|
Konstantin Lopuhin
|
bcb92b50dc
|
check that no extra kwargs are silently discarded
|
2016-02-15 19:59:47 +03:00 |
|
Konstantin Lopuhin
|
d6bea3bf2e
|
botocore not only does not allow passing our own Date header, but does not handle x-amz-date according to the spec
|
2016-02-15 19:59:47 +03:00 |
|
Konstantin Lopuhin
|
7748ee6bba
|
mock date in s3 tests when using botocore
|
2016-02-15 19:59:47 +03:00 |
|
Konstantin Lopuhin
|
c3fec83e7e
|
use botocore by default, boto is still used in "precise" env
|
2016-02-15 19:59:47 +03:00 |
|
Konstantin Lopuhin
|
1b1092b7d0
|
add Headers.to_native_string_dict - useful when interfacing with other libraries
|
2016-02-15 19:59:47 +03:00 |
|
Konstantin Lopuhin
|
467553cc29
|
fix anon test: in this case we do no signing, just change the url
|
2016-02-15 19:59:47 +03:00 |
|
Konstantin Lopuhin
|
eaf3a239e4
|
using botocore for s3 request signing: proof of concept
|
2016-02-15 19:59:46 +03:00 |
|
Paul Tremberth
|
41588397c0
|
Merge pull request #1765 from scrapy/add-deprecation-for-pydispatch
[MRG+1] Add fallback and deprecation warning for pydispatch (fixes #1762)
|
2016-02-11 19:29:53 +01:00 |
|
Elias Dorneles
|
164493df2e
|
add deprecation for pydispatch (thanks for the help @redapple)
|
2016-02-11 16:15:28 -02:00 |
|
Elias Dorneles
|
150d1c370b
|
Merge pull request #1769 from orangain/stdout-typeerror
[MRG+1] PY3: Fix TypeError when outputting to stdout
|
2016-02-10 14:30:11 -02:00 |
|
Paul Tremberth
|
c083935806
|
Merge pull request #1771 from orangain/secure-cookies
[MRG+1] PY3: Implement some attributes of WrappedRequest required in Python 3
|
2016-02-08 11:52:00 +01:00 |
|
Mikhail Korobov
|
44bc4c06ff
|
Merge pull request #1767 from orangain/sitemap-robotstxt
[MRG+1] PY3: Fix SitemapSpider to extract sitemap urls from robots.txt properly
|
2016-02-08 10:11:58 +05:00 |
|
orangain
|
1f743996ff
|
PY3: Implement some attributes of WrappedRequest required in Python 3
This will fix #1770.
|
2016-02-07 14:19:27 +09:00 |
|
orangain
|
1cb841bfb3
|
PY3: Fix TypeError when outputting to stdout
This will fix #1768.
|
2016-02-07 10:06:42 +09:00 |
|
orangain
|
25c56159b8
|
Fix SitemapSpider to extract sitemap urls from robots.txt properly
This will fix #1766.
|
2016-02-06 23:54:07 +09:00 |
|
Elias Dorneles
|
e328a9b9df
|
Merge pull request #1750 from scrapy/spider-is-idle
[MRG+1] more efficient ExecutionEngine.spider_is_idle
|
2016-02-05 21:18:35 -02:00 |
|
Elias Dorneles
|
daf0f660d1
|
Merge pull request #1662 from NicolasP/send_utf8
[MRG+1] MailSender.send: allow passing a charset.
|
2016-02-04 17:18:19 -02:00 |
|
Nicolas Pennequin
|
6efc7a9de0
|
Update the email doc for the charset argument to send().
|
2016-02-04 19:33:44 +01:00 |
|
Nicolas Pennequin
|
061c63592a
|
MailSender.send: allow passing a charset.
Resolves Issue #348
|
2016-02-04 19:33:44 +01:00 |
|
Daniel Graña
|
8f269558f1
|
Bump version: 1.2.0dev1 → 1.2.0dev2
1.2.0dev2
|
2016-02-03 22:58:21 -03:00 |
|
Daniel Graña
|
6cf4fdbae6
|
Enable travis builds on tag patterns
|
2016-02-03 22:56:49 -03:00 |
|
Elias Dorneles
|
aaccdbb49b
|
Bump version: 1.1.0rc1 → 1.2.0dev1
|
2016-02-03 21:54:39 -02:00 |
|
Elias Dorneles
|
a0b63a7707
|
Bump version: 1.1.0dev1 → 1.1.0rc1
|
2016-02-03 21:52:47 -02:00 |
|
Elias Dorneles
|
94b2c47da5
|
Merge pull request #1751 from scrapy/release-notes-1.1
[MRG+1] Release notes 1.1
|
2016-02-03 21:51:10 -02:00 |
|
Elias Dorneles
|
ca08314654
|
makes more explicit source of problems in Python 3 limitations
|
2016-02-03 21:38:25 -02:00 |
|
Elias Dorneles
|
2167573b32
|
fix comment about disabling robots.txt
|
2016-02-03 21:31:27 -02:00 |
|
Elias Dorneles
|
3180abc800
|
applying more review comments
|
2016-02-03 21:28:05 -02:00 |
|
Elias Dorneles
|
d40add7b27
|
add note about robots.txt waiting and make it explicit builtin extensions only are ported
|
2016-02-03 21:18:25 -02:00 |
|
Elias Dorneles
|
9cfefd5277
|
favor beta over basic to describe PY3 support
|
2016-02-03 20:59:56 -02:00 |
|