Pablo Hoffman
c7f8219901
- removed scrapy.conf singleton from scrapy.log, scrapy.responsetypes,
...
scrapy.http.response.text, scrapy.selector
- fixed bug with scrapy.conf.settings backwards compatibility support
- added facility to notify (and provide some guidelines) about deprecated/obsolete settings
2012-09-19 03:03:34 -03:00
Pablo Hoffman
391cc06063
Merge pull request #174 from stav/master
...
Selector documentation typo fixes
2012-09-18 11:19:23 -07:00
stav
303e13f616
selector documentation typos
2012-09-18 12:56:52 -05:00
Pablo Hoffman
3d736e657f
fixed typo in doc
2012-09-18 10:51:01 -03:00
Pablo Hoffman
eed6eb49da
make DBM the new default storage backend for HTTP cache middleware, simplified DBM storage backend code to avoid dealing with many spiders at once (not needed), and update httpcache stats names (hit -> hits, miss -> misses)
2012-09-17 10:11:07 -03:00
Pablo Hoffman
8f2dda12cc
removed another instance of scrapy.conf.settings singleton, this time from scrapy.utils.trackref. From now on, trackrefs functionality will be always enabled as it imposes a very minimal performance overhead
2012-09-16 21:21:44 -03:00
Pablo Hoffman
cd823018aa
removed another instance of scrapy.conf.settings singleton, this time from scrapy.cmdline (which manages scrapy command line tool), by moving the get_project_settings() function to scrapy.utils.project
2012-09-16 20:53:52 -03:00
Pablo Hoffman
9685c24059
restored stats_spider_{opened,closing,closed} signals in backwards compatibility mode
2012-09-14 12:38:00 -03:00
Pablo Hoffman
81ed2d2d0b
major Stats Collection refactoring: removed separation of global/per-spider stats, removed stats-related signals (stats_spider_opened, etc). Stats are much simpler now, backwards compatibility is kept on the Stats Collector API.
2012-09-14 12:31:33 -03:00
Pablo Hoffman
8b48420559
better names for scheduler stats
2012-09-13 20:07:11 -03:00
Pablo Hoffman
263e302b6a
added more scheduler stats
2012-09-13 19:46:45 -03:00
Pablo Hoffman
d6867f7984
removed sphinx warnings about duplicate reference names 'this page'
2012-09-13 15:26:45 -03:00
Pablo Hoffman
a874964ad4
renamed 'XPath Selectors' title to just 'Selectors'
2012-09-13 15:24:44 -03:00
Pablo Hoffman
acb8895e1a
changed note in scrapyd doc to use sphinx notes
2012-09-13 15:22:59 -03:00
Pablo Hoffman
26f1d5cb48
Merge pull request #171 from artem-dev/scrapyd_job_times
...
added start and stop times for scrapyd list jobs web service
2012-09-13 11:16:01 -07:00
Artem Bogomyagkov
4d8f253912
commited doc file missed from prev commit
2012-09-12 11:12:59 +03:00
Artem Bogomyagkov
5dde26d3c8
simplified code for finished jobs times, updated docs for scrapyd
2012-09-12 11:10:49 +03:00
Pablo Hoffman
7ace23c385
removed another instance of scrapy.conf singleton, this time from test_cmdline
2012-09-11 16:37:55 -03:00
Pablo Hoffman
8642645109
removed another instance of scrapy.conf singleton
2012-09-11 16:34:58 -03:00
Pablo Hoffman
7ef593c5c2
refactored MailSender to get rid of scrapy.conf singleton, also removed ill-designed scrapy.mail.mail_sent signal
2012-09-11 16:27:19 -03:00
Artem Bogomyagkov
a42a8cafef
added start and stop times for scrapyd list jobs web service
2012-09-11 21:06:36 +03:00
Pablo Hoffman
1e2efe5664
scrapy deploy: retry setup.py bdist_egg on EINTR. closes #9
2012-09-07 15:40:26 -03:00
Pablo Hoffman
3c05fbb6a9
removed redundant comment
2012-09-05 15:54:23 -03:00
Pablo Hoffman
44fe4c6b78
set missing 'settings' attribute in FeedExports
2012-09-05 15:54:13 -03:00
Pablo Hoffman
0135a5364e
removed debugging print
2012-09-05 15:53:30 -03:00
Pablo Hoffman
7bf0af7a58
fixed some bugs in get_func_args() and added more test cases
2012-09-04 19:01:20 -03:00
Pablo Hoffman
251d9a9033
backwards compatibility support for FeedExport's that don't receive a settings object in their constructor
2012-09-04 18:13:27 -03:00
Pablo Hoffman
caa64908d2
added tests for get_func_args() and support for more cases
2012-09-04 18:07:44 -03:00
Pablo Hoffman
fff2871828
added doc section (and FAQ) about spider arguments
2012-09-04 14:49:30 -03:00
Pablo Hoffman
241abcb8aa
scrapyd: log errors in API calls to the scrapyd log
2012-09-04 14:22:21 -03:00
Pablo Hoffman
f4a17ec272
removed references to Scrapy Snippets site
2012-09-03 22:19:15 -03:00
Pablo Hoffman
b901e64044
replaced memory usage acounting with (more portable) resource module, removed scrapy.utils.memory module. closes #161
2012-09-03 19:28:16 -03:00
Pablo Hoffman
e2f9daac67
fixed formatting in scrapyd release notes
2012-09-03 16:58:58 -03:00
Pablo Hoffman
dd1398b280
removed another instance of scrapy.conf.settings singleton, this time from download handlers
2012-09-03 16:44:18 -03:00
Daniel Graña
3b768ae989
oops. remove santas debug line
2012-08-31 20:10:32 -03:00
Pablo Hoffman
c6435c5aa7
restored scrapyd log message
2012-08-31 18:49:20 -03:00
Pablo Hoffman
7c8af83138
added issue role to documentation
2012-08-31 18:39:30 -03:00
Pablo Hoffman
3891c20840
Merge pull request #164 from dangra/lazy-log-formatting
...
format log lines lazily in case they are dropped by loglevels
2012-08-31 14:33:41 -07:00
Daniel Graña
a2d22307ad
update news.rst
2012-08-31 17:38:40 -03:00
Daniel Graña
dcef7b03c1
format log lines lazily in case they are dropped by loglevels
2012-08-31 17:14:35 -03:00
Pablo Hoffman
be206ca5ab
added process_start_requests method to spider middlewares
2012-08-31 16:41:50 -03:00
Pablo Hoffman
b920674666
corrected minor issue with doc references
2012-08-31 16:41:49 -03:00
Pablo Hoffman
a2d5a304b1
Merge pull request #168 from alexcepoi/parse-fixes
...
parse command: fix crash in case of bad callback
2012-08-31 09:12:19 -07:00
Alex Cepoi
0350bed8fd
parse command: fix crash in case of bad callback
2012-08-31 16:43:04 +02:00
Pablo Hoffman
4ec99117d3
fixed minor doc typo
2012-08-30 11:56:30 -03:00
Pablo Hoffman
94b40162a9
fixed tests to work on windows
2012-08-30 11:24:29 -03:00
Pablo Hoffman
3904f5f55a
more reliable way to set subprocess PYTHONPATH in tests
2012-08-30 10:09:20 -03:00
Pablo Hoffman
4a5f70278f
minor tidy up to installation guide windows notes
2012-08-29 15:44:24 -03:00
Pablo Hoffman
098d892c03
simplified installation guide to only mention pip/easy_install mechanism, and provide hints for Windows users
2012-08-29 15:37:05 -03:00
Pablo Hoffman
6217f108cc
avoid random "Interrupted system call" errors
2012-08-29 11:44:00 -03:00
