BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-24 00:23:52 +00:00
Code Releases Wiki Activity
6,401 Commits 39 Branches 119 Tags
Commit Graph

1005 Commits

Author SHA1 Message Date
Elias Dorneles
a9c69458ff Merge pull request #2422 from rolando-contrib/nested-spiders-modules 
[MRG+1] DOC State explicitly that spiders are loaded recursively.
 2016-12-07 11:21:15 -02:00
Moisés Guimarães
c08d278c0c removes note from docs. 2016-12-05 16:47:24 -03:00
Moisés Guimarães
a4178f99da fixes params types in docs. 2016-12-05 15:24:26 -03:00
Moisés Guimarães
c58ea021b8 fixes docs 2016-12-04 11:56:14 -03:00
Rolando Espinoza
6431e7a138 DOC State explicitly that spiders are loaded recursively. 2016-12-01 13:26:19 -03:00
Paul Tremberth
624284e851 Fix indent 2016-11-29 18:18:59 +01:00
Pawel Miech
f98ffb53b6 add docs for some scheduler settings 2016-11-29 16:52:54 +01:00
Paul Tremberth
059085b5b4 Remove docs for deprecated ChunkedTransfer middleware 2016-11-24 18:23:34 +01:00
Paul Tremberth
03cf5f1bd2 Remove ChunkedTransferMiddleware from default settings 2016-11-24 17:18:57 +01:00
Mikhail Korobov
a07400ce0a Merge pull request #2396 from redapple/ubuntu-packages-toc 
[MRG] DOC Remove "Ubuntu" section from sidebar/ToC
 2016-11-22 18:21:38 +05:00
Pablo Hoffman
d62776a858 mention scrapoxy in best practices doc 2016-11-16 12:19:32 -03:00
Paul Tremberth
8db8545393 Add "orphan" metadata for Ubuntu packages page 
As described in http://www.sphinx-doc.org/en/latest/markup/misc.html#metadata
 2016-11-16 13:56:58 +01:00
Mikhail Korobov
570e12b5db Merge pull request #2328 from scrapy/download-latency-meta-docs 
[MRG+1] Document `download_latency` meta key
 2016-11-14 21:24:14 +05:00
Valdir Stumm Junior
7025d6656a document download_latency meta key 2016-11-14 13:06:18 -02:00
Rahul Kant
f56aef99c2 Add closing tag in <tbody> 2016-11-07 17:49:22 +05:30
Gustavo de Andrade
f2e49bc23c Update selectors.rst 
Decode instead escape, exceptions &lt; and &amp; (kmike)
Second sentence droped (Digenis)
 2016-11-01 21:32:17 -02:00
Mikhail Korobov
9755ef933a Merge pull request #2369 from jc0n/media-pipeline-docs-typo 
Fix typo in media pipeline docs
 2016-10-29 12:46:48 +06:00
John O'Connor
d85da273be Fix typo in media pipeline docs 2016-10-28 19:44:46 -07:00
Steven Almeroth
99daea495b Doc: wording 2016-10-21 16:16:42 -07:00
Steven Almeroth
a958e54954 Doc: remove trailing spaces 2016-10-21 16:16:37 -07:00
gustavodeandrade
c7dfb5eb88 Fix issue 1704 2016-10-21 00:08:08 -02:00
Mikhail Korobov
871eec9827 Merge pull request #2327 from bopace/http-header-docs 
[MRG+1] Added documentation about accessing header values
 2016-10-20 09:05:14 +02:00
Mikhail Korobov
71c8278f57 Merge pull request #2329 from josericardo/scrapy-2262 
[MRG+1] Better explain middleware orders and processing directions
 2016-10-20 09:04:19 +02:00
bopace
fd016ee71b Fixed wording of documentation 2016-10-18 09:37:45 -06:00
Paul Tremberth
6cc83c041e Merge pull request #2330 from lfmattossch/note-python2-name 
[MRG+1] Added a note about invalid spider names in python 2 (fixes #2251)
 2016-10-18 16:35:19 +02:00
Jose Ricardo
e12e364a40 Add details to the spider middlewares docs 
Document the effects of the middleware order in a more detailed way.
 2016-10-18 12:29:30 -02:00
Luiz Fernando Mattos Schlindwein
7c33e0cb55 added a note about invalid spider names in python 2 2016-10-18 11:55:51 -02:00
Jose Ricardo
ea7bd39529 Make architecture overview references a little more clear on the docs 
Expliciting what actually happens by adding links to the respective methods
that are invoked in each processing phase.
 2016-10-18 11:50:51 -02:00
Jose Ricardo
bebcd5081c Add downloader middleware ordering details to the docs 
Add more details, making it easier to understand what are the effects of
setting a downloader middleware order.
 2016-10-18 11:22:55 -02:00
Bo Pace
bfe28ae707 Added documentation about accessing header values 2016-10-17 14:10:05 -06:00
Bernardas
dfba151f59 Remove unnecessary note for the JsonWriterPipeline example 2016-10-05 16:36:02 +00:00
Bernardas
eb91cb8ea2 fix JsonWriterPipeline example 2016-10-03 20:31:41 +00:00
Wieland Hoffmann
e8edc6c2bb Remove duplicate colons from the feed export settings docs 2016-10-02 16:09:29 +02:00
Paul Tremberth
fed53c1e28 Merge pull request #2267 from scrapy/deprecate-ubuntu-packages 
[MRG+1] Deprecate official Ubuntu packages and update installation instructions
 2016-09-30 17:35:29 +02:00
Elias Dorneles
8bafcf33a1 improve description of engine 2016-09-30 11:20:30 -03:00
Elias Dorneles
e107212043 restore explanation loop to step 1 (see comment in 3ac3ac4) 2016-09-29 19:02:57 -03:00
Elias Dorneles
86eb865598 Merge pull request #2287 from pawelmhm/docs-2230 
[MRG+1][docs/item_pipeline] process_item returning Deferred docs
 2016-09-29 09:15:09 -03:00
Elias Dorneles
3ac3ac4d92 docs: update data flow description and image (fixes: #2278) 
This fixes the explanation to use Requests instead of URLs,
which is what actually happens, and is also consistent with the
new tutorial, which already explains how URLs become Request objects.

I've also changed the "loop", jumping from 9 to step 2.
 2016-09-28 16:38:45 -03:00
pawelmhm
39dbd890b0 [docs/item_pipeline] process_item returning Deferred docs 
* quote url
* use hash of url as filename
 2016-09-24 08:36:09 +02:00
Pawel Miech
b2bfd1e5c5 [docs] document that process_item can return Deferred 2016-09-23 10:36:03 +02:00
Paul Tremberth
e59d79bf37 Add note on "to" and "cc" as lists for sending emails 
Fixes GH-2244
 2016-09-20 17:18:49 +02:00
Ashish Kulkarni
8893527a2f temporarily deprecate official Ubuntu packages 
They are not currently updated and fail to install on
Ubuntu 16.04. Also update the instructions to refer to
the earliest supported LTS (Ubuntu 12.04).

fixes #2137 and closes #2076
 2016-09-20 11:10:59 +02:00
Elias Dorneles
063315258e Merge pull request #2202 from scrapy/doc-arch-overview2 
[MRG+1] DOC move Data Flow below the picture; add links to components
 2016-09-19 08:11:18 -03:00
Matti Remes
0ef570f6f0 Update exceptions.rst 
Added the missing dot. (+1 squashed commit)
Squashed commits:
[2198972] Update exceptions.rst

There are namely no constructors in classes in Python but an ``__init__`` method instead.
 2016-09-08 19:38:17 +05:00
Paul Tremberth
eedb6ce774 Merge pull request #2190 from stummjr/fix-docs 
[MRG+1] Fix RANDOMIZE_DOWNLOAD_DELAY description in the docs
 2016-08-31 11:51:47 +02:00
Mikhail Korobov
495d322691 DOC move Data Flow below the picture; add links to components 2016-08-26 20:16:22 +05:00
Thom Dixon
f68dc3026d Fix indentation 2016-08-24 09:11:27 -07:00
Thom Dixon
633abfbea1 Correct documentation about Response parameters 
This fixes issue #2196
 2016-08-24 08:47:52 -07:00
Valdir Stumm Junior
d61650d843 fix RANDOMIZE_DOWNLOAD_DELAY description in the docs 2016-08-19 18:24:32 -03:00
Ashish Kulkarni
bb3b806467 Use w3lib.url.canonicalize_url() from w3lib 1.15.0 
Also remove code/imports which are now unused due to this change.

fixes #2157
 2016-08-16 17:42:16 +05:30