Hobson Lane
85a80d0752
remove "for brevity's sake" line and correct "Torrent item"
...
Torrent item -> TorrentItem class
2014-01-15 17:29:23 -08:00
Hobson Lane
a3db95985b
another import name correction by pablo
2014-01-14 21:04:15 -08:00
Hobson Lane
6ba0857a5c
documentation code example correction corrections per pablohoffman
2014-01-10 10:37:27 -08:00
Mikhail Korobov
a27d91f0a6
Rename BaseSpider to Spider. See GH-495.
2013-12-30 19:46:41 +06:00
RasPat1
ff21281b95
Note about selector class import
...
This is the salient point of this code compared to the last example. We have a selector now and this is how we use it. Especially since the user has just come from the shell where the pre-instantiated selector is taken for granted.
2013-12-15 13:46:42 -05:00
Pablo Hoffman
f2741c413e
fix method name in tutorial. closes GH-480
2013-12-02 13:24:12 -02:00
Daniel Graña
2df8156431
Drop Python 2.6 support
2013-10-29 13:44:00 -02:00
Pablo Hoffman
e8ee449a2a
Merge pull request #432 from darkrho/crawl-url
...
Removed URL reference in crawl command and .tld suffix in docs for spider names
2013-10-21 09:40:58 -07:00
Rolando Espinoza La fuente
34543c2b2e
DOCS removed .tld suffix for spider names for the sake of consistency.
2013-10-19 23:03:20 -04:00
Daniel Graña
155ea08ea1
use sel name for Selector's instances in docs, internals and shell
2013-10-15 15:58:42 -02:00
Daniel Graña
1abb1af0c6
fix typos and wording on selector's introduction
2013-10-15 10:13:43 -02:00
Daniel Graña
4645f9e03c
Updates docs to reflect unified selectors api
2013-10-14 16:31:20 -02:00
Pablo Hoffman
e1683ddf9b
fix doc typo
2013-10-09 17:24:12 -02:00
Pablo Hoffman
b1d1a36a1e
add note about enclosing urls with quotes when running from command-line. closes GH-384
2013-09-18 18:01:28 -03:00
Kumara Tharmalingam
bbb0603091
Fixed directory location for dmoz_spider.py file
...
It should be under 'tutorial/spiders' not 'dmoz/spiders'
2013-09-15 21:55:52 -07:00
Daniel Graña
0400b18efa
docs: list lxml as installation prerequisite
2013-09-09 12:44:26 -03:00
Hart
c00c4d7148
correction to description of example XPath retrieval in overview doc
2013-08-03 17:08:58 -07:00
Juan M Uys
4de3aa4932
Update overview.rst
2013-04-08 14:13:15 +02:00
Pablo Hoffman
51c301b3a2
added link to python binary libs, for windows installation
2013-03-13 03:18:33 -03:00
Pablo Hoffman
22edc44c6c
doc: remove links to diveintopython.org, which is no longer available. closes #246
2013-02-14 11:09:40 -02:00
Pablo Hoffman
d6867f7984
removed sphinx warnings about duplicate reference names 'this page'
2012-09-13 15:26:45 -03:00
Pablo Hoffman
f4a17ec272
removed references to Scrapy Snippets site
2012-09-03 22:19:15 -03:00
Pablo Hoffman
4a5f70278f
minor tidy up to installation guide windows notes
2012-08-29 15:44:24 -03:00
Pablo Hoffman
098d892c03
simplified installation guide to only mention pip/easy_install mechanism, and provide hints for Windows users
2012-08-29 15:37:05 -03:00
Daniel Graña
abcc8c9f63
Recommend pypi as single way to install on Windows
2012-08-06 10:21:13 -03:00
Valentin-Costel Hăloiu
00bfb37e79
Update master
2012-07-04 06:55:01 +03:00
Pablo Hoffman
2fb5e62c39
doc: update overview page to point to the genspider command. refs #107
2012-04-19 02:37:22 -03:00
Pablo Hoffman
4f28ffcb2c
removed no longer needed dependency on simplejson
2012-04-10 16:01:36 -03:00
Pablo Hoffman
6e8edbd72e
switched default selectors backend to lxml
2012-04-10 15:52:14 -03:00
Pablo Hoffman
b6ae266546
Removed (very old and possibly broken) backwards compatibility support for Twisted 2.5
2012-03-15 00:28:24 -03:00
Pablo Hoffman
e521da2e2f
Dropped support for Python 2.5. See: http://blog.scrapy.org/scrapy-dropping-support-for-python-25
2012-03-01 08:18:12 -02:00
Pablo Hoffman
0be421fbf0
fixed reference to tutorial directory
2011-12-23 18:57:11 -02:00
Daniel Graña
bcb31988f2
change tutorial to follow changes on dmoz site
2011-12-14 13:03:31 -02:00
Pablo Hoffman
ade5efdc61
added -o option to scrapy crawl, a convenient shortcut for using feed exports
2011-10-22 20:53:49 -02:00
Pablo Hoffman
431441cb52
updated documentation to remove references to old issue tracker and mercurial repos
2011-09-25 13:06:24 -03:00
Pablo Hoffman
76af0cdd44
updated documentation and code to use -s instead of --set option
2011-09-01 14:35:37 -03:00
Pablo Hoffman
a3697421c0
some minor updates to documentation
2011-08-11 09:19:59 -03:00
Pablo Hoffman
5da6ffb57b
Automated merge with ssh://hg.scrapy.org:2222/scrapy-0.12
2011-08-11 09:11:19 -03:00
Pablo Hoffman
bc2d2183e9
fixed import in doc
2011-08-11 09:11:08 -03:00
Pablo Hoffman
c59340150f
Added cached DNS resolver based on old caching resolver extension from scrapy.contrib.resolver. This new one is *not* an extension, it comes builtin and always enabled.
2011-07-27 03:45:15 -03:00
Pablo Hoffman
57c43fdce6
added SitemapSpider, with tests and doc
2011-06-15 11:54:34 -03:00
Pablo Hoffman
5bf733b6f6
Changed default representation of items to pretty-printed dicts. This improves
...
default logging by making log more readable in the default case, for both Scraped and Dropped lines.
Projects can still customize how items are represented by overriding the item's __str__ method, as usual.
2011-06-03 01:13:01 -03:00
Pablo Hoffman
951ba507f9
Removed support for default values in Scrapy items, which have proven confusing in the past
2011-05-19 21:42:46 -03:00
Pablo Hoffman
503f302010
removed remaining references to scheduler middleware from doc, as it will be removed on next release
2011-05-18 19:48:48 -03:00
Pablo Hoffman
7f97259ba7
added w3lib to requirements, in installation guide
2011-05-01 11:14:57 -03:00
Pablo Hoffman
bb2b67c862
updated tutorial to use 'dmoz' as the name of the spider instead of 'dmoz.org', so that it's more similar to the dirbot example project
2011-04-28 09:31:57 -03:00
Pablo Hoffman
bf73002428
removed googledir example, replaced by dirbot project on github. updated docs accordingly
2011-04-28 02:28:39 -03:00
Pablo Hoffman
181d1c09ae
Fixed typo and code indentation in the doc. Closes #307 and #308
2011-02-09 11:19:46 -02:00
Pablo Hoffman
426b6fa100
docs/intro/install.rst: added -U flag to easy_install command
2010-11-22 13:50:19 -02:00
Pablo Hoffman
ac007802d6
Simplified installation guide, including lxml as alternative dependency to libxml2. Closes #280
2010-11-17 21:32:23 -02:00
