1
0
mirror of https://github.com/scrapy/scrapy.git synced 2025-03-15 04:50:33 +00:00

242 Commits

Author SHA1 Message Date
Mikhail Korobov
54d106033f
Merge pull request #3649 from Gallaecio/inline-code-formatting
[MRG+1] Switch from ` to `` where inline code formatting is desired
2019-03-14 22:18:57 +05:00
Anubhav Patel
82049e9c41 make suggested changes. 2019-03-10 20:17:07 +05:30
Adrián Chaves
e108e3adbf Clarify the documentation of DEPTH_PRIORITY further 2019-03-08 15:13:11 +01:00
Anubhav Patel
924b67437b move api docs to source code 2019-03-07 16:40:59 +05:30
Adrián Chaves
75d6f56c8a Switch from to ` where inline code formatting is desired 2019-03-01 16:56:58 +01:00
Mikhail Korobov
71e47629b1 DOC fix docs for AWS_... settings. A follow-up to GH-2609. 2018-12-26 16:35:05 +05:00
Mikhail Korobov
0ccead6681 DOC more Python 3 in examples 2018-09-12 17:57:27 +05:00
Ryan P Kilby
2dfc5d128b Update DEPTH_STATS refs to DEPTH_STATS_VERBOSE 2018-05-17 11:00:21 -04:00
Daniel Graña
2dee191374
Merge branch 'master' into extending-s3-files-store 2017-12-31 16:44:38 -03:00
Aditya
df7e0a4315 Use https link in default user agent 2017-10-28 23:37:44 +05:30
Aditya
9dd680d5c9 Use https for external links wherever possible in docs 2017-10-26 23:33:45 +05:30
Mikhail Korobov
083880888b DOC fixed rst syntax in DOWNLOAD_FAIL_ON_DATALOSS docs 2017-05-30 00:44:11 +05:00
Paul Tremberth
8aa2e4f997 Merge pull request #1829 from nyov/nyov/editor
[MRG+1] Remove dependency on os.environ from default settings
2017-05-18 21:32:51 +02:00
Eli Atzaba
b74b98fa3e cleanup: removed unused MEMUSAGE_REPORT
Signed-off-by: Eli Atzaba <eliat123@gmail.com>
2017-05-16 13:59:58 +03:00
Tiago Cardoso
00ee9eaeaf Mention how to disable request filtering in documentation of DUPEFILTER_CLASS setting 2017-04-22 14:36:44 +01:00
Oto Brglez
11cf6ad425 Comments for AWS_ENDPOINT_URL setting. 2017-03-19 12:48:06 +01:00
Paul Tremberth
a7f5207e9f Update version added for SPIDER_LOADER_WARN_ONLY 2017-03-10 16:38:09 +01:00
Paul Tremberth
9628a73972 Update settings docs for new SPIDER_LOADER_WARN_ONLY 2017-03-09 17:40:34 +01:00
Rolando Espinoza
f01ae6ffcd Handle data loss gracefully.
Websites that return a wrong ``Content-Length`` header may cause a data
loss error. Also when a chunked response is not finished properly.

This change adds a new setting ``DOWNLOAD_FAIL_ON_DATALOSS`` (default:
``True``) and request.meta key ``download_fail_on_dataloss``.
2017-03-01 11:43:53 -03:00
Mikhail Korobov
93e449f1f6 Merge pull request #2343 from redapple/anonymous-ftp
[MRG+1] Support Anonymous FTP
2017-02-20 23:19:54 +06:00
Paul Tremberth
f3a7567443 Add note on FTP_PASSWORD default value 2017-02-20 17:15:05 +01:00
Paul Tremberth
d35a01a103 Update default password 2017-02-20 14:23:23 +01:00
Paul Tremberth
b80e1bb6c5 Document new FTP_* settings 2017-02-20 14:19:36 +01:00
Mikhail Korobov
85a124970a Enable memusage extension by default. Fixes GH-2187. 2017-02-07 03:32:54 +05:00
nyov
2240f00a13 Remove dependency on os.environ from default settings
Avoid loading settings from environment in scrapy core.
Instead it's better to populate them from the starting
shell or an embedding script.
2016-12-26 20:49:16 +00:00
Mikhail Korobov
0fc73a9d55 DOC update examples with long longger names 2016-12-16 21:47:58 +05:00
Mikhail Korobov
05cec0f2f3 fixed ReST syntax 2016-12-09 00:04:04 +05:00
Akhil Lb
a75ad2bbc6 LOG_SHORT_NAMES option 2016-12-09 00:04:04 +05:00
Mikhail Korobov
bd8c293a97 Merge pull request #2411 from redapple/remove-chunked-transfer-mw
[MRG+1] Remove ChunkedTransferMiddleware from default settings
2016-12-07 22:05:15 +05:00
Paul Tremberth
624284e851 Fix indent 2016-11-29 18:18:59 +01:00
Pawel Miech
f98ffb53b6 add docs for some scheduler settings 2016-11-29 16:52:54 +01:00
Paul Tremberth
03cf5f1bd2 Remove ChunkedTransferMiddleware from default settings 2016-11-24 17:18:57 +01:00
Valdir Stumm Junior
d61650d843 fix RANDOMIZE_DOWNLOAD_DELAY description in the docs 2016-08-19 18:24:32 -03:00
Elias Dorneles
34e7dadf38 Merge pull request #1610 from darshanime/scheduler_debug
[MGR+1] Change, document `LOG_UNSERIALIZABLE_REQUESTS`
2016-07-29 10:12:52 -03:00
darshanime
d8e62e660d update log demo print 2016-07-26 20:46:12 +05:30
darshanime
0c77b6d033 update docs for settings 2016-07-25 18:05:51 +05:30
Darshan Chaudhary
472a8a47d0 Change name, log once 2016-07-25 18:03:43 +05:30
Jesús Losada
c6a2ca4e93 Document S3 capabilities in FilesPipeline and ImagesPipeline 2016-07-22 19:32:29 +02:00
Raul Gallegos
49ac7de231 prioritize default headers over user agent 2016-07-05 15:38:17 -05:00
Paul Tremberth
9b3c72cb37 DOC Place FEED_TEMPDIR setting at lexicographical position 2016-05-17 16:25:57 +02:00
Paul Tremberth
bf7f675493 Merge pull request #1847 from aron-bordin/add_blocking_storage_path_setting
[MRG+2] added BLOCKING_FEED_STORAGE_PATH to settings
2016-04-01 15:47:06 +02:00
Aron Bordin
9250a5bffa added FEED_TEMPDIR to settings 2016-04-01 00:05:21 -03:00
Aron Bordin
2cfe9e424d small doc style fixes 2016-03-05 19:54:06 -03:00
nyov
5876b9aa30 Update documentation links 2016-03-03 16:28:33 +00:00
Paul Tremberth
174f526709 Document DOWNLOADER_* settings for HTTP/1.0 and TLS 2016-02-26 18:23:57 +01:00
Konstantin Lopuhin
6137dd96d9 Fix documentation for S3_STORE_ACL (now settings.FILES_STORE_S3_ACL) settings: it has nothing to do with feed exporters. 2016-02-24 10:16:41 +03:00
Konstantin Lopuhin
164f300762 See #1778 - change default S3 ACL to "private" and allow customization via settings 2016-02-24 10:16:41 +03:00
Paul Tremberth
44d8df2060 Add versionadded note for MEMUSAGE_CHECK_INTERVAL_SECONDS 2016-02-03 12:35:26 +01:00
Paul Tremberth
d999e3f7a7 More explicit description of DEPTH_PRIORITY 2016-01-27 12:57:03 +01:00
Paul Tremberth
6ed08d2332 Add note for DEPTH_PRIORITY 2016-01-27 11:53:29 +01:00