Rolando Espinoza
0cc1d870db
docs: minor tidy up sample code and missing shell prompts.
2013-09-30 09:58:10 -04:00
Loren Davie
8af0e89e85
Corrected typo.
2013-09-29 17:06:46 -04:00
Loren Davie
f49f5724d5
Added dynamic creation of item classes to practices.rst.
2013-09-28 09:00:48 -04:00
irgmedeiros
9b50409986
Update the second code example
...
Update the second code example to reflect the last change in the first example.
2013-09-27 18:22:33 -03:00
irgmedeiros
d9e0fdc9aa
Update practices.rst
...
With this modification scrapy runs the spider with project settings. The previous example ran only with default settings resulting in ignoring all user settings as pipelines for example.
2013-09-27 17:56:30 -03:00
Daniel Graña
265910aae6
Merge pull request #363 from taikano/sitemap_alternate
...
also fetch alternate URLs from sitemaps, see #360
2013-09-26 09:15:02 -07:00
Pablo Hoffman
12280c2a95
fix sphinx references in doc
2013-09-25 15:13:17 -03:00
Pablo Hoffman
fc388f4636
Make ITEM_PIPELINE setting a dict
...
This is for consistency with how spider and downloader middlewares are
defined. ITEM_PIPELINE_BASE was also added and both remain empty.
Backwards compatibility is kept (with a warning) with list-based
ITEM_PIPELINES.
2013-09-23 17:50:43 -03:00
Pablo Hoffman
b1d1a36a1e
add note about enclosing urls with quotes when running from command-line. closes GH-384
2013-09-18 18:01:28 -03:00
cacovsky
71b320914a
Update request-response.rst
...
Fix small doc typo (too many backticks)
2013-09-18 11:45:25 -03:00
Kumara Tharmalingam
bbb0603091
Fixed directory location for dmoz_spider.py file
...
It should be under 'tutorial/spiders' not 'dmoz/spiders'
2013-09-15 21:55:52 -07:00
Daniel Graña
0400b18efa
docs: list lxml as installation prerequisite
2013-09-09 12:44:26 -03:00
Stefan
6994959181
renamed to sitemap_alternate_links and added default value, see #360
2013-09-08 10:38:28 +02:00
Stefan
8ed2d0cda1
improved changes to allow retrieval of alternate links in sitemaps, see #360
2013-09-07 12:56:30 +02:00
Daniel Graña
e6b3ca0180
Add 0.18.2 release notes
2013-09-03 14:29:40 -03:00
Daniel Graña
0f00b1602a
merge 0.18 release notes
2013-08-27 18:47:46 -03:00
Pablo Hoffman
19ff9ac4f9
url/body attributes of Request/Response objects are now immutable
2013-08-23 12:43:22 -03:00
Pablo Hoffman
86230c0ab8
added quantal & raring to support ubuntu releases
2013-08-22 21:49:55 -03:00
Mikhail Korobov
034ffae60f
Recommend Pillow instead of PIL. Closes GH-317.
2013-08-18 00:44:01 +06:00
Berend Iwema
32b6364bcd
#327 - Support STARTTLS / SSL option in email sender
2013-08-14 12:59:01 +02:00
Pablo Hoffman
c0b26e3d49
minor updates to 0.18 release notes
2013-08-14 01:39:44 -03:00
Daniel Graña
ed5b9068d2
fix contributters list format
2013-08-12 11:26:16 -03:00
Daniel Graña
a6693c9a5c
updated release notes and bumped version to 0.18.0
2013-08-09 19:02:28 -03:00
Hart
c00c4d7148
correction to description of example XPath retrieval in overview doc
2013-08-03 17:08:58 -07:00
Hart
0ad01c34d4
fixed typo to parallel fix on 0.16 branch
2013-08-03 17:06:10 -07:00
Rocio Aramberri
d227d530f6
Added COMPRESSION_ENABLED setting to enable or disable the HttpCompressionMiddleware
...
Added COMPRESSION_ENABLE setting to docs
Added COMPRESSION_ENABLED setting to default settings
2013-08-01 11:31:28 -03:00
Dan
1ca31244b0
Fixed ordering of super argument call.
2013-07-16 14:50:10 -04:00
Dan
e12b689c4f
Updated documentation of spider arguments to include required super call.
2013-07-16 14:26:53 -04:00
Mikhail Korobov
1a1c93fafe
tiny FormRequest doc fix
2013-07-15 15:47:34 +06:00
Mikhail Korobov
ac2fadf3ab
DownloaderMiddleware.process_response docs fix
...
"returns an exception" -> "raises an exception"
2013-07-08 19:41:58 +06:00
Mikhail Korobov
39e5da5f66
improve docs for DownloaderMiddleware.process_response
2013-07-08 19:17:29 +06:00
Pablo Hoffman
0f4b70f582
remove no deprecated request_scheduled signal
...
It will be replaced by more accurate scheduler signals (proposal will
come soon)
2013-06-27 11:23:24 -03:00
nramirezuy
bef8ade956
removed request_received and added request_scheduled
2013-06-26 16:45:46 -03:00
Pablo Hoffman
819b2776dd
Merge pull request #326 from berendiwema/master
...
Include example of how to stop the reactor from script
2013-06-25 13:30:07 -07:00
nramirezuy
83b2774354
remove wrong default httpcache
2013-06-25 17:01:29 -03:00
Berend Iwema
aec314db09
added a bit more documentation on how to close the reactor when running scrapy from a script
2013-06-25 16:08:22 +02:00
Pablo Hoffman
bbde1d0e0b
Merge pull request #275 from stav/doc
...
doc: Response.replace() cannot take meta argument
2013-06-24 11:09:28 -07:00
Capi Etheriel
50fa46d183
Document CrawlSpider.parse_start_urls method
2013-06-09 04:03:20 -03:00
Daniel Graña
b4fca90bba
merge 0.16.5 release notes
2013-05-30 18:49:00 -03:00
cacovsky
8007762890
Add FAQ entry referencing Request.meta usage
2013-05-27 13:02:17 -03:00
Pablo Hoffman
845c64b89d
add benchmarking to 0.18 release notes
2013-05-17 10:38:42 -03:00
Pablo Hoffman
ca12886acb
update copyright notes
2013-05-16 15:05:52 -03:00
Pablo Hoffman
8e49fed918
minor improvements to benchmarking doc
2013-05-16 13:23:13 -03:00
Pablo Hoffman
76087e336a
add scrapy bench command for benchmarking, with documentation
2013-05-16 13:15:25 -03:00
Pablo Hoffman
66311db23e
mention crawlera in best practices, as a way to deal with bans
2013-05-04 18:20:23 -03:00
Pablo Hoffman
9361c89573
remove scrapyd doc, as it was moved to its own repo
2013-04-27 04:15:42 -03:00
Pablo Hoffman
d02da2f31f
ported code to use queuelib
2013-04-23 17:48:09 -03:00
Pablo Hoffman
7a1536f76e
Merge pull request #290 from nramirezuy/item-copy
...
added copy method to item
2013-04-19 09:27:44 -07:00
Nicolás Ramírez
6df274bba5
added copy method to item
2013-04-19 13:23:53 -03:00
Mikhail Korobov
b245d592aa
Update faq.rst
...
spider.DOWNLOAD_DELAY is deprecated
2013-04-18 02:42:15 +06:00
