Pablo Hoffman
8f4c879b58
extended documentation on how to access crawler stats from extensions
2012-10-25 11:28:23 -02:00
Daniel Graña
a9616f387e
fix dashes under rst headers
2012-10-18 20:14:30 -02:00
Daniel Graña
4852c6102d
set release date for 0.16.0 in news
2012-10-18 20:07:36 -02:00
Pablo Hoffman
1a905d62f5
removed scrapy.log.started attribute, and avoid checking if log has already been started (since it should be called once anyway)
2012-10-09 16:05:19 -02:00
Pablo Hoffman
1f89eb59fe
fixed doc reference to topics-contracts
2012-10-09 16:02:12 -02:00
Pablo Hoffman
9e4c6aba2f
updated release notes with new stats recently added
2012-10-09 12:50:42 -02:00
Pablo Hoffman
7458092eef
added spider contracts to release notes and warn that its API is still subject to change
2012-09-29 03:06:30 -03:00
Pablo Hoffman
c380910b40
Merge pull request #167 from alexcepoi/sep-017
...
Spider contracts (SEP-017)
2012-09-28 13:57:07 -07:00
Pablo Hoffman
a5f8943d3e
added change introduced in previous commit to release notes
2012-09-26 13:17:01 -03:00
Alex Cepoi
11d29c7005
SEP-017 contracts: add tests and minor improvements
2012-09-21 00:12:46 +02:00
Pablo Hoffman
b46b5a6ef0
Documented AutoThrottle extension and added to extensions available by
...
default. Also deprecated concurrency and delay settings, in favour of
using the standard Scrapy ones.
2012-09-20 18:52:57 -03:00
Pablo Hoffman
c7f8219901
- removed scrapy.conf singleton from scrapy.log, scrapy.responsetypes,
...
scrapy.http.response.text, scrapy.selector
- fixed bug with scrapy.conf.settings backwards compatibility support
- added facility to notify (and provide some guidelines) about deprecated/obsolete settings
2012-09-19 03:03:34 -03:00
stav
303e13f616
selector documentation typos
2012-09-18 12:56:52 -05:00
Pablo Hoffman
3d736e657f
fixed typo in doc
2012-09-18 10:51:01 -03:00
Pablo Hoffman
eed6eb49da
make DBM the new default storage backend for HTTP cache middleware, simplified DBM storage backend code to avoid dealing with many spiders at once (not needed), and update httpcache stats names (hit -> hits, miss -> misses)
2012-09-17 10:11:07 -03:00
Pablo Hoffman
8f2dda12cc
removed another instance of scrapy.conf.settings singleton, this time from scrapy.utils.trackref. From now on, trackrefs functionality will be always enabled as it imposes a very minimal performance overhead
2012-09-16 21:21:44 -03:00
Pablo Hoffman
9685c24059
restored stats_spider_{opened,closing,closed} signals in backwards compatibility mode
2012-09-14 12:38:00 -03:00
Pablo Hoffman
81ed2d2d0b
major Stats Collection refactoring: removed separation of global/per-spider stats, removed stats-related signals (stats_spider_opened, etc). Stats are much simpler now, backwards compatibility is kept on the Stats Collector API.
2012-09-14 12:31:33 -03:00
Pablo Hoffman
d6867f7984
removed sphinx warnings about duplicate reference names 'this page'
2012-09-13 15:26:45 -03:00
Pablo Hoffman
a874964ad4
renamed 'XPath Selectors' title to just 'Selectors'
2012-09-13 15:24:44 -03:00
Pablo Hoffman
acb8895e1a
changed note in scrapyd doc to use sphinx notes
2012-09-13 15:22:59 -03:00
Pablo Hoffman
26f1d5cb48
Merge pull request #171 from artem-dev/scrapyd_job_times
...
added start and stop times for scrapyd list jobs web service
2012-09-13 11:16:01 -07:00
Artem Bogomyagkov
4d8f253912
commited doc file missed from prev commit
2012-09-12 11:12:59 +03:00
Pablo Hoffman
7ef593c5c2
refactored MailSender to get rid of scrapy.conf singleton, also removed ill-designed scrapy.mail.mail_sent signal
2012-09-11 16:27:19 -03:00
Alex Cepoi
bf8dc61fb7
SEP-017 contracts: pretty-printing and docs
2012-09-10 23:17:27 +02:00
Pablo Hoffman
fff2871828
added doc section (and FAQ) about spider arguments
2012-09-04 14:49:30 -03:00
Pablo Hoffman
f4a17ec272
removed references to Scrapy Snippets site
2012-09-03 22:19:15 -03:00
Pablo Hoffman
b901e64044
replaced memory usage acounting with (more portable) resource module, removed scrapy.utils.memory module. closes #161
2012-09-03 19:28:16 -03:00
Pablo Hoffman
e2f9daac67
fixed formatting in scrapyd release notes
2012-09-03 16:58:58 -03:00
Pablo Hoffman
dd1398b280
removed another instance of scrapy.conf.settings singleton, this time from download handlers
2012-09-03 16:44:18 -03:00
Pablo Hoffman
7c8af83138
added issue role to documentation
2012-08-31 18:39:30 -03:00
Daniel Graña
a2d22307ad
update news.rst
2012-08-31 17:38:40 -03:00
Pablo Hoffman
be206ca5ab
added process_start_requests method to spider middlewares
2012-08-31 16:41:50 -03:00
Pablo Hoffman
b920674666
corrected minor issue with doc references
2012-08-31 16:41:49 -03:00
Pablo Hoffman
4ec99117d3
fixed minor doc typo
2012-08-30 11:56:30 -03:00
Pablo Hoffman
4a5f70278f
minor tidy up to installation guide windows notes
2012-08-29 15:44:24 -03:00
Pablo Hoffman
098d892c03
simplified installation guide to only mention pip/easy_install mechanism, and provide hints for Windows users
2012-08-29 15:37:05 -03:00
Pablo Hoffman
70f8e517a1
promoted DjangoItem to main contrib
2012-08-29 11:23:11 -03:00
Pablo Hoffman
da234af456
Merge branch 'singleton_removal'
2012-08-28 18:42:33 -03:00
Pablo Hoffman
babfc6e79b
Updated documentation after singleton removal changes.
...
Also removed some unused code and made some minor additional
refactoring.
2012-08-28 18:35:57 -03:00
Pablo Hoffman
5fa5c4545b
use ScrapyJSONEncoder JsonItemExporter & JsonLinesItemExporter, to support nested items properly
2012-08-22 13:46:50 -03:00
Pablo Hoffman
36f47a4aec
Removed per-spider settings concept, and scrapy.conf.settings singleton from many extensions and middlewares. There are some still remaining, that will be removed in future commits
2012-08-21 17:27:45 -03:00
Daniel Graña
abcc8c9f63
Recommend pypi as single way to install on Windows
2012-08-06 10:21:13 -03:00
Daniel Graña
b7b0c49520
append parse command to example code sections in docs. closes #162
2012-08-06 09:10:16 -03:00
Pablo Hoffman
832e45073b
fixed typo in stats documentation. closes #159
2012-07-20 17:13:06 -03:00
Valentin-Costel Hăloiu
00bfb37e79
Update master
2012-07-04 06:55:01 +03:00
Daniel Graña
277ed0ae23
Merge pull request #145 from alexcepoi/cookies-changes
...
domain and path support for request cookies
2012-06-25 11:29:04 -07:00
Alexandru Cepoi
177c81745d
domain and path support for request cookies
2012-06-25 20:17:59 +02:00
Pablo Hoffman
179e3810dc
fixed links to doc. closes #150
2012-06-24 01:00:33 -03:00
Alexandru Cepoi
f4faa19e31
added docs topic debugging spiders
2012-06-21 20:03:33 +02:00
