BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-25 00:04:03 +00:00
Code Releases Wiki Activity
5,769 Commits 39 Branches 119 Tags
Commit Graph

861 Commits

Author SHA1 Message Date
Matti Remes
0ef570f6f0 Update exceptions.rst 
Added the missing dot. (+1 squashed commit)
Squashed commits:
[2198972] Update exceptions.rst

There are namely no constructors in classes in Python but an ``__init__`` method instead.
 2016-09-08 19:38:17 +05:00
Paul Tremberth
eedb6ce774 Merge pull request #2190 from stummjr/fix-docs 
[MRG+1] Fix RANDOMIZE_DOWNLOAD_DELAY description in the docs
 2016-08-31 11:51:47 +02:00
Thom Dixon
f68dc3026d Fix indentation 2016-08-24 09:11:27 -07:00
Thom Dixon
633abfbea1 Correct documentation about Response parameters 
This fixes issue #2196
 2016-08-24 08:47:52 -07:00
Valdir Stumm Junior
d61650d843 fix RANDOMIZE_DOWNLOAD_DELAY description in the docs 2016-08-19 18:24:32 -03:00
Ashish Kulkarni
bb3b806467 Use w3lib.url.canonicalize_url() from w3lib 1.15.0 
Also remove code/imports which are now unused due to this change.

fixes #2157
 2016-08-16 17:42:16 +05:30
Lorena
7d432872bf text updates to match graphic 2016-08-04 11:01:14 -07:00
Lorena
04f93e096c updated graph for organization/clarity 2016-08-04 10:04:47 -07:00
Elias Dorneles
34e7dadf38 Merge pull request #1610 from darshanime/scheduler_debug 
[MGR+1] Change, document `LOG_UNSERIALIZABLE_REQUESTS`
 2016-07-29 10:12:52 -03:00
darshanime
d8e62e660d update log demo print 2016-07-26 20:46:12 +05:30
Paul Tremberth
fe68a45c82 Merge pull request #2138 from jesuslosada/master 
[MRG+1] Apply the FILES_STORE_S3_ACL setting in ImagesPipeline
 2016-07-26 12:50:25 +02:00
Elias Dorneles
b6375d3701 Merge pull request #1566 from darshanime/signal-example 
[MRG+1] Include example for signal docs
 2016-07-25 10:49:29 -03:00
Darshan Chaudhary
a2e6452554 Include signal example 2016-07-25 18:41:25 +05:30
darshanime
0c77b6d033 update docs for settings 2016-07-25 18:05:51 +05:30
Darshan Chaudhary
472a8a47d0 Change name, log once 2016-07-25 18:03:43 +05:30
Jesús Losada
8221556089 Add documentation on Amazon S3 ACLs 2016-07-22 23:22:18 +02:00
Jesús Losada
c6a2ca4e93 Document S3 capabilities in FilesPipeline and ImagesPipeline 2016-07-22 19:32:29 +02:00
Paul Tremberth
ec1c61504a Merge pull request #2005 from feliperuhland/master 
[MRG+1] Included new optional parameter in startproject command line
 2016-07-19 12:31:06 +02:00
Kenneth Schnall
5fabed514e Update logging.rst 2016-07-15 23:14:11 -04:00
Mikhail Korobov
79639d0fec Merge pull request #1989 from pawelmhm/fix-images-pipeline-uppercase-other 
[MRG+1] [image_pipeline] bring back uppercase class attributes
 2016-07-13 14:44:00 +00:00
Kenneth Schnall
2489f84d39 Update practices.rst 2016-07-12 15:34:50 -04:00
Paul Tremberth
c3109daa72 Merge pull request #2034 from dracony/master 
[MRG+1] Added option to turn off ensure_ascii for JSON exporters
 2016-07-12 17:01:09 +02:00
Dracony
33a39b368f added FEED_EXPORT_ENCODING setting to allow encoding specification 2016-07-12 16:20:17 +02:00
Pawel Miech
c22cc1096b [image_pipeline] style edits 
* 80 characters line limit
* shortening some code
* removed dead code
* add doctest for _key_for_pipe function
 2016-07-12 13:58:36 +02:00
Valdir Stumm Junior
081595a2e4 document new genspider behavior 2016-07-06 15:10:48 -03:00
Raul Gallegos
49ac7de231 prioritize default headers over user agent 2016-07-05 15:38:17 -05:00
Pawel Miech
10b79c9b3e [files-pipeline] update docs with note about settings 
for subclasses.
 2016-06-15 15:49:11 +02:00
Pawel Miech
a62d4b081c [image-pipeline] image settings with class name 
allow to have image settings with class name, so that settings for user defined ImagePipeline
subclasses can be defined easily.
 2016-06-10 12:48:02 +02:00
Konstantin Lopuhin
da8d0eadd6 Use "url" variable in the example 
Instead of hardcoded http://www.example.com: without it url variable is unused and only one request will make it past dupefilter.
 2016-05-27 10:47:57 +03:00
Felipe Ruhland
089483aece
Updated docs for new option in command line 2016-05-23 23:16:15 -03:00
Paul Tremberth
b3367c7acd DOC Add info and example on errbacks 2016-05-18 18:00:09 +02:00
Pawel Miech
6c67db3917 [image_pipeline] tests for class attrs backward compatibility 
and docs about image pipeline settings.
 2016-05-18 12:04:52 +02:00
Paul Tremberth
9b3c72cb37 DOC Place FEED_TEMPDIR setting at lexicographical position 2016-05-17 16:25:57 +02:00
Panayiotis Lipiridis
3f8be374c3 Github >> GitHub 2016-05-09 16:15:18 +02:00
nblock
a3557dd34d Fix spelling mistake 2016-04-11 14:06:57 +02:00
Paul Tremberth
0ede017d2a Merge pull request #1891 from djunzu/update_files_images_pipelines 
[MRG+1] Change Files/ImagesPipelines class attributes to instance attributes
 2016-04-08 12:55:09 +02:00
djunzu
6988e9cd4b Update docs. 
modified:   docs/topics/media-pipeline.rst
 2016-04-01 21:51:15 -03:00
Paul Tremberth
bf7f675493 Merge pull request #1847 from aron-bordin/add_blocking_storage_path_setting 
[MRG+2] added BLOCKING_FEED_STORAGE_PATH to settings
 2016-04-01 15:47:06 +02:00
Aron Bordin
9250a5bffa added FEED_TEMPDIR to settings 2016-04-01 00:05:21 -03:00
Paul Tremberth
9d8c368ce8 Merge pull request #1879 from scrapy/scrapy-arch-docs 
DOC improved Architecture overview
 2016-03-31 12:09:24 +02:00
Paul Tremberth
1075587dbd Add support for Sphinx 1.4 
See http://www.sphinx-doc.org/en/stable/changes.html#release-1-4-released-mar-28-2016

sphinx_rtd_theme has become optional, needs to be added to reqs

https://github.com/sphinx-doc/sphinx/pull/2320 changes node entries tuples
to 5 values instead of 4

`sh` syntax highlighting added very locally in selectors.rst
because of this warning/error with Sphinx 1.4:

```
Warning, treated as error:
/home/paul/src/scrapy/docs/topics/selectors.rst:743:
WARNING: Could not lex literal_block as "python". Highlighting skipped.
```
 2016-03-30 14:40:52 +02:00
Lele
7082454f2a Changed sel. to response. for clarity 
Changed sel. to response. to comply with the rest of the examples in the same section, to avoid confusion.
 2016-03-28 05:27:15 +05:00
Mikhail Korobov
4f335b5a01 DOC clarify Architecture docs 2016-03-25 17:03:41 +05:00
Mikhail Korobov
3ca977a8cb DOC improved Architecture overview 
* spiders don't have to work on specific domains;
* explain what to use Downloader middleware for
  and what to use Spider middleware for;
* Engine no longer locates spiders based on domains;
* "Spider middleware output direction" step was missing.

See also: GH-1569.
 2016-03-25 07:11:33 +05:00
Aron Bordin
2cfe9e424d small doc style fixes 2016-03-05 19:54:06 -03:00
nyov
5876b9aa30 Update documentation links 2016-03-03 16:28:33 +00:00
Paul Tremberth
174f526709 Document DOWNLOADER_* settings for HTTP/1.0 and TLS 2016-02-26 18:23:57 +01:00
Konstantin Lopuhin
6137dd96d9 Fix documentation for S3_STORE_ACL (now settings.FILES_STORE_S3_ACL) settings: it has nothing to do with feed exporters. 2016-02-24 10:16:41 +03:00
Konstantin Lopuhin
164f300762 See #1778 - change default S3 ACL to "private" and allow customization via settings 2016-02-24 10:16:41 +03:00
Konstantin Lopuhin
e7c4806c5e Update feedstorage docs: add botocore, mention that boto is supported only on Python 2 2016-02-15 19:59:48 +03:00