BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-25 07:03:56 +00:00
Code Releases Wiki Activity
5,769 Commits 39 Branches 119 Tags
Commit Graph

5769 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jesús Losada
f72991a96b Add the IMAGES_STORE_S3_ACL setting 2016-07-22 19:18:13 +02:00
Paul Tremberth
ec1c61504a Merge pull request #2005 from feliperuhland/master 
[MRG+1] Included new optional parameter in startproject command line
 2016-07-19 12:31:06 +02:00
Felipe Ruhland
fe088925a3 Included implementation notes in docstring 
Signed-off-by: Felipe Ruhland <felipe.ruhland@gmail.com>
 2016-07-19 00:12:39 -03:00
Felipe Ruhland
de64a1f68a Fix scrapy.cfg validation 
Signed-off-by: Felipe Ruhland <felipe.ruhland@gmail.com>
 2016-07-19 00:04:45 -03:00
Elias Dorneles
a86562b72b Merge pull request #2128 from kas/patch-1 
(docs) Minor grammar fix in logging.rst
 2016-07-18 09:24:03 -03:00
Kenneth Schnall
5fabed514e Update logging.rst 2016-07-15 23:14:11 -04:00
Mikhail Korobov
79639d0fec Merge pull request #1989 from pawelmhm/fix-images-pipeline-uppercase-other 
[MRG+1] [image_pipeline] bring back uppercase class attributes
 2016-07-13 14:44:00 +00:00
Pawel Miech
ceecf3b26c [image_pipeline] minor style tweaks 2016-07-13 16:19:24 +02:00
Mikhail Korobov
2dd1a9e3bc Merge pull request #2094 from redapple/dns-invalid-id 
Catch and ignore certification verification exception for IP-address hosts
 2016-07-13 10:48:08 +00:00
Paul Tremberth
005cf949b8 Change wording of warning + docstring for ScrapyClientTLSOptions 2016-07-13 11:01:30 +02:00
Elias Dorneles
3b5915568b Merge pull request #2120 from kas/kas-patch-1 
(docs) Fix small grammar issue in practices.rst (Supersedes #2119)
 2016-07-12 17:05:39 -03:00
Kenneth Schnall
2489f84d39 Update practices.rst 2016-07-12 15:34:50 -04:00
Paul Tremberth
859bcf4820 Rephrase warning 2016-07-12 17:53:19 +02:00
Paul Tremberth
c3109daa72 Merge pull request #2034 from dracony/master 
[MRG+1] Added option to turn off ensure_ascii for JSON exporters
 2016-07-12 17:01:09 +02:00
Dracony
33a39b368f added FEED_EXPORT_ENCODING setting to allow encoding specification 2016-07-12 16:20:17 +02:00
Pawel Miech
9818c97f6e [image_pipeline] dont use assert in doctest 2016-07-12 14:15:41 +02:00
Pawel Miech
c22cc1096b [image_pipeline] style edits 
* 80 characters line limit
* shortening some code
* removed dead code
* add doctest for _key_for_pipe function
 2016-07-12 13:58:36 +02:00
Paul Tremberth
8a22a74e48 Merge pull request #2065 from redapple/octet-stream-no-decompress 
[MRG+2] [HttpCompressionMW] Do not decompress gzip binary/octet-stream responses
 2016-07-11 12:15:38 +02:00
Paul Tremberth
962eb11c73 Simplify regex conditions 2016-07-11 11:30:05 +02:00
Paul Tremberth
778f1cf84c Merge remote-tracking branch 'origin/master' into octet-stream-no-decompress 2016-07-08 18:13:20 +02:00
Elias Dorneles
d43a35735a Merge pull request #2050 from Tethik/is_gzipped_fix 
[MRG+1] Is_gzipped for application/x-gzip;charset=utf-8
 2016-07-08 08:47:56 -03:00
Mikhail Korobov
b7553d921a Merge pull request #2038 from redapple/canonicalize-idna-failures 
[MRG] Do not fail on canonicalizing URLs with wrong netlocs
 2016-07-08 10:47:54 +06:00
Mikhail Korobov
5f261cf295 Merge pull request #2026 from Digenis/contributing-full_subjects 
[MRG+1] Encourage complete titles in pull requests
 2016-07-08 10:45:22 +06:00
Mikhail Korobov
52a52e2388 Merge pull request #2001 from matveinazaruk/issue-2000 
[MRG+1] Fixed choosing of response class.
 2016-07-08 10:44:24 +06:00
Elias Dorneles
0ab7c1fbf9 Merge pull request #2052 from stummjr/genspider-standalone 
[MRG+1] Enable genspider command outside project folder
 2016-07-06 18:01:30 -03:00
Valdir Stumm Junior
081595a2e4 document new genspider behavior 2016-07-06 15:10:48 -03:00
Valdir Stumm Junior
8987b17730 remove references to Item classes in templates 2016-07-06 15:10:48 -03:00
Valdir Stumm Junior
1779f5feca enable genspider command outside projects 2016-07-06 15:10:48 -03:00
Mikhail Korobov
759a555d28 Merge pull request #2069 from redapple/https-connect-host 
[MRG] Add "Host" header in CONNECT requests to HTTPS proxies
 2016-07-06 21:43:41 +06:00
Paul Tremberth
1737c15826 Merge pull request #2091 from eLRuLL/default_headers_priority 
[MRG+1] prioritize default headers over user agent middlewares
 2016-07-06 17:29:39 +02:00
Paul Tremberth
15d0c89159 Cleanup unused argument 2016-07-06 17:15:21 +02:00
Mikhail Korobov
8ec6c22d60 Merge pull request #2095 from scrapy/testing-deps 
[MRG+1] TST pin pytest-cov to 2.2.1; upgrade pytest
 2016-07-06 20:07:19 +06:00
Mikhail Korobov
4273734744 TST pin pytest-cov to 2.2.1; upgrade pytest 2016-07-06 18:29:49 +05:00
Paul Tremberth
37efdde3e3 Catch and ignore TLS verification exception for IP-address hosts 
Fixes GH-2092
 2016-07-06 14:20:13 +02:00
Raul Gallegos
49ac7de231 prioritize default headers over user agent 2016-07-05 15:38:17 -05:00
Elias Dorneles
441df4c853 Merge pull request #2060 from pawelmhm/windows-python3 
[MRG+1] [docs] add note about windows + python3
 2016-07-05 10:26:33 -03:00
Νικόλαος-Διγενής Καραγιάννης
2a92ffb409 Encourage descriptive PR titles 2016-07-05 14:21:28 +03:00
Pawel Miech
b67440dec0 docs on Scrapy on Windows + Python 3 2016-07-04 16:37:51 +02:00
Paul Tremberth
6539277f99 Fix CONNECT request timeout (with an ugly hack) 2016-06-21 17:14:41 +02:00
Paul Tremberth
d9343463cb Add "Host" header in CONNECT requests to HTTPS proxies 2016-06-21 13:26:32 +02:00
Paul Tremberth
10a2c46e12 [HttpCompressionMiddleware] Do not decompress binary/octet-stream responses 2016-06-20 16:37:00 +02:00
Pawel Miech
fa4d0cdfe5 [FilesPipeline, ImagesPipeline] fix for cls attrs with DEFAULT prefix 
some class attributes for ImagePipeline and FilesPipeline had DEFAULT prefix. These
attributes should be preserved as well, if users subclasses define values for
DEFAULT_<CLS_ATTRIBUTE_NAME> attribute this value should be preserved.
 2016-06-20 12:53:20 +02:00
Pawel Miech
07d1605586 [docs] warnings about windows + python 3 in faq and install 2016-06-17 13:28:51 +02:00
Pawel Miech
73cc066caa [docs] add note about windows + python3 2016-06-17 09:21:58 +02:00
Pawel Miech
10b79c9b3e [files-pipeline] update docs with note about settings 
for subclasses.
 2016-06-15 15:49:11 +02:00
Pawel Miech
539d34bce0 [media-pipeline, file-pipeline] allow setting custom settings for subclasses 
* move key_for_pipe function to media pipeline so that file pipeline can use it
* use key_for_pipe in file pipeline so that users can define custom settings for subclasses easily
* add tests for file pipelines attributes and settings
 2016-06-15 15:39:11 +02:00
Pawel Miech
acbfdc6184 [files_pipeline] ensure class attributes are preserved 
dont override class attributes with default settings (same as in image pipeline).
 2016-06-15 15:14:28 +02:00
Pawel Miech
c6d1686d98 [files_pipeline] unify tests for files pipeline 
if test tests same thing but for different field it can be unified into one.
 2016-06-15 14:48:25 +02:00
Pawel Miech
72e4d5f33e [image_pipeline] another test for subclass inheritance 
test case when subclass inherits from base class and has no attributes nor
settings defined.
 2016-06-15 14:07:17 +02:00
Pawel Miech
ee39d11e45 [image_pipeline] refactor and simplify tests for image settings 
unify tests that test same thing for different attribute values into one. Add
better docstrings for tests.
 2016-06-15 11:25:38 +02:00