457 Commits

Author	SHA1	Message	Date
Paul Tremberth	da36b7d386	Merge pull request #1761 from lopuhin/py3-s3-botocore ... [MRG+1] Py3 S3 botocore	2016-02-18 15:11:03 +01:00
Konstantin Lopuhin	d61fbcc8b5	Support headers in S3FilesStore.persist_file for botocore	2016-02-18 10:57:02 +03:00
Paul Tremberth	cabed6f183	More liberal Content-Disposition header parsing ... Fixes #1782	2016-02-17 16:55:28 +01:00
Konstantin Lopuhin	d1ecb8cd38	Fix S3TestCase for precise env: we reraise TypeError as NotConfigured in this case	2016-02-15 19:59:48 +03:00
Konstantin Lopuhin	77ebb13684	fix assertRaises for precise env	2016-02-15 19:59:48 +03:00
Konstantin Lopuhin	08bc41cc68	py3: reviewed s3 downloader handlers	2016-02-15 19:59:48 +03:00
Konstantin Lopuhin	3cb7a567ea	py3 fix for TestS3FilesStore: checksum is a native string	2016-02-15 19:59:48 +03:00
Konstantin Lopuhin	cfc567f48e	botocore support for S3FilesStore	2016-02-15 19:59:48 +03:00
Konstantin Lopuhin	32cd8c9165	add direct test for S3FilesStore	2016-02-15 19:59:48 +03:00
Konstantin Lopuhin	d1470e85a2	S3FeedStorageTest: pass on py3, add some non-ascii content to be sure	2016-02-15 19:59:48 +03:00
Konstantin Lopuhin	3ada45a9bb	S3FeedStorageTest: add botocore support, and organize boto/botocore checks	2016-02-15 19:59:47 +03:00
Konstantin Lopuhin	5d2f067458	S3FeedStorageTest: delete key after test	2016-02-15 19:59:47 +03:00
Konstantin Lopuhin	19b2910ad1	Fix assert_aws_environ: check for botocore with boto fallback on PY2	2016-02-15 19:59:47 +03:00
Konstantin Lopuhin	bcb92b50dc	check that no extra kwargs are silently discarded	2016-02-15 19:59:47 +03:00
Konstantin Lopuhin	d6bea3bf2e	botocore not only does not allow passing our own Date header, but does not handle x-amz-date according to the spec	2016-02-15 19:59:47 +03:00
Konstantin Lopuhin	7748ee6bba	mock date in s3 tests when using botocore	2016-02-15 19:59:47 +03:00
Konstantin Lopuhin	c3fec83e7e	use botocore by default, boto is still used in "precise" env	2016-02-15 19:59:47 +03:00
Konstantin Lopuhin	467553cc29	fix anon test: in this case we do no signing, just change the url	2016-02-15 19:59:47 +03:00
Konstantin Lopuhin	eaf3a239e4	using botocore for s3 request signing: proof of concept	2016-02-15 19:59:46 +03:00
Paul Tremberth	41588397c0	Merge pull request #1765 from scrapy/add-deprecation-for-pydispatch ... [MRG+1] Add fallback and deprecation warning for pydispatch (fixes #1762)	2016-02-11 19:29:53 +01:00
Elias Dorneles	164493df2e	add deprecation for pydispatch (thanks for the help @redapple)	2016-02-11 16:15:28 -02:00
Paul Tremberth	c083935806	Merge pull request #1771 from orangain/secure-cookies ... [MRG+1] PY3: Implement some attributes of WrappedRequest required in Python 3	2016-02-08 11:52:00 +01:00
orangain	1f743996ff	PY3: Implement some attributes of WrappedRequest required in Python 3 ... This will fix #1770.	2016-02-07 14:19:27 +09:00
orangain	25c56159b8	Fix SitemapSpider to extract sitemap urls from robots.txt properly ... This will fix #1766.	2016-02-06 23:54:07 +09:00
Nicolas Pennequin	061c63592a	MailSender.send: allow passing a charset. ... Resolves Issue #348	2016-02-04 19:33:44 +01:00
Elias Dorneles	a8a6f050e7	Merge pull request #1735 from ArturGaspar/master ... [MRG+1] Fix for KeyError in robots.txt middleware	2016-02-03 13:30:06 -02:00
Mikhail Korobov	43a53aca12	Merge pull request #1746 from redapple/shell-settings-logging ... [MRG+1] Remove __str__ and __repr__ from settings, introduce copy_to_dict()	2016-02-03 19:59:24 +05:00
stummjr	bb2cf7c0d7	Fixed bug on XMLItemExporter with non-string fields in items	2016-01-30 10:00:06 -02:00
Paul Tremberth	d843a0aae8	Amend "settings" command to output JSON for dict settings	2016-01-29 21:12:03 +01:00
Paul Tremberth	aa78758bc7	Update tests for settings copy_to_dict()	2016-01-29 18:59:12 +01:00
Mikhail Korobov	a35aec71e9	Merge pull request #1710 from redapple/1550-shell_file-cont ... [MRG+1] shell command's ability to open local files + tests	2016-01-29 21:03:40 +05:00
Paul Tremberth	78f00401cd	Remove unused import in tests	2016-01-29 16:56:05 +01:00
stummjr	cf2ebb0687	Include tests for exporters: JSON, JSON-Lines, Pickle and Marshal.	2016-01-28 19:43:53 -02:00
Paul Tremberth	c6f374f2eb	Merge remote-tracking branch 'origin/master' into 1550-shell_file-cont	2016-01-28 14:02:48 +01:00
Paul Tremberth	481e251775	Move guess_scheme() tests to relevant test module	2016-01-28 13:51:50 +01:00
Paul Tremberth	cae268402d	Move guess_scheme() to scrapy.utils.url	2016-01-28 13:42:04 +01:00
stummjr	c55ff110a3	Fix CSV exporter for non string Python types.	2016-01-27 15:43:17 -02:00
stummjr	f1d971a5c0	fix PythonItemExporter for non-string types	2016-01-27 14:34:46 -02:00
Artur Gaspar	dc8701ea42	Add test for already failed deferreds when downloading page in robots.txt middleware.	2016-01-27 12:56:42 -02:00
Mikhail Korobov	4bcbb77bcc	response.text. Fixes GH-1729.	2016-01-27 01:28:11 +05:00
Paul Tremberth	29695375d1	Add test for raised exception with SpiderState extension when no JOBDIR used	2016-01-26 16:33:24 +01:00
Νικόλαος-Διγενής Καραγιάννης	1cffa99e0d	tests+doc for subdomains in offsite middleware	2016-01-26 12:49:43 +02:00
Daniel Graña	a7b86137d0	Merge pull request #1499 from scrapy/py3-port-exporters ... [MRG+1] PY3 exporters	2016-01-26 01:01:30 -03:00
Daniel Graña	d0eacfe0f9	Add test case for marshal item exporter	2016-01-26 00:26:27 -03:00
Elias Dorneles	2dfdde3c79	fallback to repr when can't convert to native string	2016-01-25 22:24:35 -02:00
Elias Dorneles	23b3336c1f	add test for invalid option	2016-01-25 22:11:04 -02:00
Daniel Graña	b20463183c	Merge pull request #1716 from lopuhin/py3-test-pipeline-files-images ... [MRG+1] py3: test_pipeline_files and test_pipeline_images	2016-01-25 18:24:12 -03:00
Elias Dorneles	0c44fac2b5	added tests for feed export marshal and pickle	2016-01-24 19:17:42 -02:00
Konstantin Lopuhin	1be90323c2	py3: properly skip s3 tests on py3	2016-01-24 23:44:56 +03:00
Konstantin Lopuhin	4233b3cda4	py3: reviewed passing test_spidermiddleware_httperror.py	2016-01-24 23:10:03 +03:00