Paul Tremberth
bf7f675493
Merge pull request #1847 from aron-bordin/add_blocking_storage_path_setting
...
[MRG+2] added BLOCKING_FEED_STORAGE_PATH to settings
2016-04-01 15:47:06 +02:00
Aron Bordin
9250a5bffa
added FEED_TEMPDIR to settings
2016-04-01 00:05:21 -03:00
Paul Tremberth
9d8c368ce8
Merge pull request #1879 from scrapy/scrapy-arch-docs
...
DOC improved Architecture overview
2016-03-31 12:09:24 +02:00
Paul Tremberth
1075587dbd
Add support for Sphinx 1.4
...
See http://www.sphinx-doc.org/en/stable/changes.html#release-1-4-released-mar-28-2016
sphinx_rtd_theme has become optional, needs to be added to reqs
https://github.com/sphinx-doc/sphinx/pull/2320 changes node entries tuples
to 5 values instead of 4
`sh` syntax highlighting added very locally in selectors.rst
because of this warning/error with Sphinx 1.4:
```
Warning, treated as error:
/home/paul/src/scrapy/docs/topics/selectors.rst:743:
WARNING: Could not lex literal_block as "python". Highlighting skipped.
```
2016-03-30 14:40:52 +02:00
Lele
7082454f2a
Changed sel. to response. for clarity
...
Changed sel. to response. to comply with the rest of the examples in the same section, to avoid confusion.
2016-03-28 05:27:15 +05:00
Mikhail Korobov
4f335b5a01
DOC clarify Architecture docs
2016-03-25 17:03:41 +05:00
Mikhail Korobov
3ca977a8cb
DOC improved Architecture overview
...
* spiders don't have to work on specific domains;
* explain what to use Downloader middleware for
and what to use Spider middleware for;
* Engine no longer locates spiders based on domains;
* "Spider middleware output direction" step was missing.
See also: GH-1569.
2016-03-25 07:11:33 +05:00
Aron Bordin
2cfe9e424d
small doc style fixes
2016-03-05 19:54:06 -03:00
nyov
5876b9aa30
Update documentation links
2016-03-03 16:28:33 +00:00
Paul Tremberth
90c64a6d13
Update release notes about change of default S3 ACL policy to "private"
2016-02-29 12:27:25 +01:00
Paul Tremberth
709b4fa884
Update release notes about HTTPS downloader
2016-02-26 18:35:29 +01:00
Paul Tremberth
174f526709
Document DOWNLOADER_* settings for HTTP/1.0 and TLS
2016-02-26 18:23:57 +01:00
Paul Tremberth
84dea19455
Update release notes
2016-02-25 00:59:15 +01:00
Konstantin Lopuhin
6137dd96d9
Fix documentation for S3_STORE_ACL (now settings.FILES_STORE_S3_ACL) settings: it has nothing to do with feed exporters.
2016-02-24 10:16:41 +03:00
Konstantin Lopuhin
164f300762
See #1778 - change default S3 ACL to "private" and allow customization via settings
2016-02-24 10:16:41 +03:00
Konstantin Lopuhin
e7c4806c5e
Update feedstorage docs: add botocore, mention that boto is supported only on Python 2
2016-02-15 19:59:48 +03:00
Elias Dorneles
164493df2e
add deprecation for pydispatch (thanks for the help @redapple)
2016-02-11 16:15:28 -02:00
Nicolas Pennequin
6efc7a9de0
Update the email doc for the charset argument to send().
2016-02-04 19:33:44 +01:00
Elias Dorneles
ca08314654
makes more explicit source of problems in Python 3 limitations
2016-02-03 21:38:25 -02:00
Elias Dorneles
2167573b32
fix comment about disabling robots.txt
2016-02-03 21:31:27 -02:00
Elias Dorneles
3180abc800
applying more review comments
2016-02-03 21:28:05 -02:00
Elias Dorneles
d40add7b27
add note about robots.txt waiting and make it explicit builtin extensions only are ported
2016-02-03 21:18:25 -02:00
Elias Dorneles
9cfefd5277
favor beta over basic to describe PY3 support
2016-02-03 20:59:56 -02:00
Elias Dorneles
2f697ce59f
added note about how to revert to old behavior for robotstxt
2016-02-03 20:51:26 -02:00
Elias Dorneles
89a088b3c3
applying review comments
2016-02-03 20:45:55 -02:00
Paul Tremberth
972f192568
Blank line before CoC
2016-02-03 22:58:49 +01:00
Paul Tremberth
728e505e53
Merge remote-tracking branch 'origin/master' into release-notes-1.1
2016-02-03 22:47:15 +01:00
Valdir Stumm Jr
dd7b0eb359
Updates to release notes
2016-02-03 19:23:02 -02:00
Paul Tremberth
51dc741fa2
Add 1.0.5 release notes
2016-02-03 21:07:21 +01:00
Valdir Stumm Jr
0f816524e8
edit API changes, Deprecations, and Relocations sections.
2016-02-03 16:21:32 -02:00
Paul Tremberth
fd3193da86
Reword some of "New Features and Enhancements"
2016-02-03 18:19:38 +01:00
Paul Tremberth
3b2a6abc0b
Format "Code refactoring" subsection
2016-02-03 17:37:58 +01:00
Paul Tremberth
ed02455010
Format "Logging" subsection
2016-02-03 17:27:43 +01:00
Valdir Stumm Jr
1fd95c9c9f
fixed references to issues and fit text to 80 cols in the first two sections
2016-02-03 12:45:48 -02:00
Valdir Stumm Jr
a21c90bee7
edits on Py3 Support and New features sections
2016-02-03 11:54:46 -02:00
Paul Tremberth
44d8df2060
Add versionadded note for MEMUSAGE_CHECK_INTERVAL_SECONDS
2016-02-03 12:35:26 +01:00
Paul Tremberth
2b033eebce
Fix recently added HTTPCACHE_ settings versionadded notes
2016-02-03 12:34:44 +01:00
Paul Tremberth
142aa21737
Add AUTOTHROTTLE_TARGET_CONCURRENCY versionadded note
2016-02-03 12:33:10 +01:00
Paul Tremberth
db0697bc06
Add 1.1 release notes (draft)
2016-02-03 12:32:40 +01:00
Paul Tremberth
c6f374f2eb
Merge remote-tracking branch 'origin/master' into 1550-shell_file-cont
2016-01-28 14:02:48 +01:00
Paul Tremberth
27fb200e45
Merge pull request #1727 from redapple/priority-adjust
...
[MRG+1] Clarify priority adjust settings docs
2016-01-27 18:52:39 +01:00
Mikhail Korobov
7ca9ae1976
DOC typo fix
2016-01-27 17:54:28 +05:00
Paul Tremberth
d999e3f7a7
More explicit description of DEPTH_PRIORITY
2016-01-27 12:57:03 +01:00
Paul Tremberth
6ed08d2332
Add note for DEPTH_PRIORITY
2016-01-27 11:53:29 +01:00
Mikhail Korobov
4bcbb77bcc
response.text. Fixes GH-1729.
2016-01-27 01:28:11 +05:00
Paul Tremberth
1c83108893
Clarify priority adjust settings docs
...
Fixes #1593
2016-01-26 19:24:11 +01:00
Paul Tremberth
bb1f4013a3
Rewrite warning about shell with local files as note
2016-01-26 17:23:28 +01:00
Mikhail Korobov
f30758c246
Enable robots.txt handling by default for new projects. Fixes GH-1668.
...
For backwards compatibility reasons the default value is not changed.
2016-01-26 17:47:46 +05:00
Νικόλαος-Διγενής Καραγιάννης
1cffa99e0d
tests+doc for subdomains in offsite middleware
2016-01-26 12:49:43 +02:00
Paul Tremberth
713e1eee9b
Update docs about local files support for "scrapy shell"
2016-01-26 10:44:38 +01:00
