Elias Dorneles
86eb865598
Merge pull request #2287 from pawelmhm/docs-2230
...
[MRG+1][docs/item_pipeline] process_item returning Deferred docs
2016-09-29 09:15:09 -03:00
Elias Dorneles
60559369b5
Merge pull request #2292 from stummjr/fix-example-section
...
[MRG+1] Update examples section in documentation with quotes.toscrape spiders
2016-09-29 09:11:45 -03:00
Elias Dorneles
3ac3ac4d92
docs: update data flow description and image ( fixes : #2278 )
...
This fixes the explanation to use Requests instead of URLs,
which is what actually happens, and is also consistent with the
new tutorial, which already explains how URLs become Request objects.
I've also changed the "loop", jumping from 9 to step 2.
2016-09-28 16:38:45 -03:00
Valdir Stumm Junior
5680c61063
Doc: update examples section with quotes.toscrape.com spiders
2016-09-28 15:45:01 -03:00
pawelmhm
39dbd890b0
[docs/item_pipeline] process_item returning Deferred docs
...
* quote url
* use hash of url as filename
2016-09-24 08:36:09 +02:00
Paul Tremberth
80c1e5dc25
Set release date, fix typo and add tutorial improvement issue number
2016-09-23 16:11:50 +02:00
Paul Tremberth
a0f87d2f45
Update release notes for upcoming 1.1.3 bugfix release
2016-09-23 16:11:50 +02:00
Pawel Miech
b2bfd1e5c5
[docs] document that process_item can return Deferred
2016-09-23 10:36:03 +02:00
Paul Tremberth
a975a50558
Merge pull request #2252 from eliasdorneles/tutorial-upgrades
...
[MRG+2] Tutorial: rewrite tutorial seeking to improve learning path
2016-09-22 16:39:21 +02:00
Elias Dorneles
f4a2208916
addressing review comments and other minor editing
2016-09-22 11:04:45 -03:00
Paul Tremberth
2e08a9b412
Merge pull request #2271 from redapple/mailsender-lists
...
[MRG+1] Add note on "to" and "cc" as lists for sending emails
2016-09-22 12:00:14 +02:00
Elias Dorneles
d636e5baa8
better description for start_requests expected return value
2016-09-21 18:54:12 -03:00
Elias Dorneles
32017a76f8
recommend learn python the hard way for beginners
2016-09-21 11:06:36 -03:00
Elias Dorneles
38266cc949
recommend Dive into Python and Python tutorial instead of LPTHW for non-beginners
2016-09-21 11:02:24 -03:00
Elias Dorneles
c126c59361
address more review comments
2016-09-20 18:19:25 -03:00
Elias Dorneles
a876ea5bd2
minor grammar fix
2016-09-20 15:10:49 -03:00
Elias Dorneles
bc41fdf20e
address review comments, add debug log to initial spider
2016-09-20 15:04:08 -03:00
Paul Tremberth
40293551b2
Remove mention of odd-numbered versions for development releases
...
Fixes GH-1317
2016-09-20 18:15:45 +02:00
Elias Dorneles
125b691102
more reviewing and editing, minor restructure, syntax fixes
2016-09-20 12:47:03 -03:00
Paul Tremberth
e59d79bf37
Add note on "to" and "cc" as lists for sending emails
...
Fixes GH-2244
2016-09-20 17:18:49 +02:00
Elias Dorneles
8975371a57
Merge branch 'master' into tutorial-upgrades
2016-09-20 09:45:05 -03:00
Elias Dorneles
f4f93c5c26
fix tox docs build, adjust title
2016-09-20 09:19:59 -03:00
Valdir Stumm Junior
fee07835f2
Completing the data extraction section
2016-09-19 19:19:44 -03:00
Elias Dorneles
2a409d1d95
[wip] changing introduction to scraping with selectors
2016-09-19 17:13:04 -03:00
Elias Dorneles
063315258e
Merge pull request #2202 from scrapy/doc-arch-overview2
...
[MRG+1] DOC move Data Flow below the picture; add links to components
2016-09-19 08:11:18 -03:00
Wayne Lovely
cc8497abb1
Fix a dict key in the tutorial
2016-09-17 11:09:28 +00:00
Valdir Stumm Junior
233b98d642
include section describing spider arguments
2016-09-16 18:08:10 -03:00
Elias Dorneles
31545a9f84
tutorial: updating extracting data section to introduce CSS and XPath equally
2016-09-16 17:13:24 -03:00
Elias Dorneles
147e75602d
update after review comments (thanks @stummjr)
2016-09-16 16:47:24 -03:00
Elias Dorneles
31260cf02f
mentions stackoverflow as help channel ( fixes #2255 )
2016-09-16 16:05:36 -03:00
Elias Dorneles
21de617c77
mention that spiders need to subclass scrapy.Spider
2016-09-16 15:55:14 -03:00
Elias Dorneles
b2a5cddbb0
tutorial: update section about following links, expand examples
...
adding an AuthorSpider to demonstrate further a different crawling
arrangement.
2016-09-16 15:49:49 -03:00
Valdir Stumm Junior
0cd9dfcc85
small fixes on tutorial
2016-09-16 15:21:49 -03:00
Valdir Stumm Junior
0da497cf7a
updates on the first section (our first spider)
2016-09-16 11:55:23 -03:00
Elias Dorneles
c508f40689
use harcoded URLs, remove item reference on second spider
2016-09-15 18:05:09 -03:00
Elias Dorneles
2427791287
tutorial: remove item class definition and present start_requests first
...
This changes the tutorial, removing the step of creating an item class
and also starts by presenting the start_requests method instead of
start_urls.
2016-09-15 17:46:31 -03:00
Elias Dorneles
75531e409e
use better condition in example spider
2016-09-15 16:56:13 -03:00
Elias Dorneles
1d159ae6f9
minor grammar fix
2016-09-15 15:37:03 -03:00
Elias Dorneles
18bd0b0886
docs: update overview spider code to use toscrape.com and minor changes
...
So, this will replace the spider example code from the overview that
scrapes questions from StackOverflow by a spider scraping quotes (much
like the one in the tutorial), and upates the text around it to be
consistent.
There are also minor wording changes plus a small Sphinx/reST syntax fix
on the features list at the bottom (it was creating a definition list,
causing one line to be bold).
2016-09-15 15:16:30 -03:00
Paul Tremberth
2f60f2a5a6
Merge pull request #2236 from stummjr/new-tutorial-toscrape
...
[MRG+1] Update broken Scrapy tutorial to use quotes.toscrape.com
2016-09-15 12:05:03 +02:00
Elias Dorneles
a9a96bed8f
updated tutorial as per review comments
2016-09-14 18:09:39 -03:00
Valdir Stumm Junior
bc67cd9edd
fix indentation issue
2016-09-14 12:39:29 -03:00
Valdir Stumm Junior
10f8c52f5d
changed tutorial examples from dmoz to quotes.toscrape.com
2016-09-13 14:05:26 -03:00
Matti Remes
0ef570f6f0
Update exceptions.rst
...
Added the missing dot. (+1 squashed commit)
Squashed commits:
[2198972] Update exceptions.rst
There are namely no constructors in classes in Python but an ``__init__`` method instead.
2016-09-08 19:38:17 +05:00
Valdir Stumm Junior
9cea6f0730
Add Segment Analytics to Documentation
2016-09-02 14:51:07 -03:00
Paul Tremberth
eedb6ce774
Merge pull request #2190 from stummjr/fix-docs
...
[MRG+1] Fix RANDOMIZE_DOWNLOAD_DELAY description in the docs
2016-08-31 11:51:47 +02:00
Mikhail Korobov
495d322691
DOC move Data Flow below the picture; add links to components
2016-08-26 20:16:22 +05:00
Thom Dixon
f68dc3026d
Fix indentation
2016-08-24 09:11:27 -07:00
Thom Dixon
633abfbea1
Correct documentation about Response parameters
...
This fixes issue #2196
2016-08-24 08:47:52 -07:00
Valdir Stumm Junior
d61650d843
fix RANDOMIZE_DOWNLOAD_DELAY description in the docs
2016-08-19 18:24:32 -03:00
