Shadab Zafar
5a58d64131
Fix some redirection links in documentation
...
Fixes #606
2015-03-18 19:41:26 -03:00
Pablo Hoffman
9361c89573
remove scrapyd doc, as it was moved to its own repo
2013-04-27 04:15:42 -03:00
Chris Tilden
aae6aed4fb
fixes spelling errors in documentation
2013-01-22 14:52:18 -08:00
Pablo Hoffman
acb8895e1a
changed note in scrapyd doc to use sphinx notes
2012-09-13 15:22:59 -03:00
Artem Bogomyagkov
4d8f253912
commited doc file missed from prev commit
2012-09-12 11:12:59 +03:00
Pablo Hoffman
fff2871828
added doc section (and FAQ) about spider arguments
2012-09-04 14:49:30 -03:00
Pablo Hoffman
e2f9daac67
fixed formatting in scrapyd release notes
2012-09-03 16:58:58 -03:00
Pablo Hoffman
58e88ed246
scrapyd: do not set SCRAPY_FEED_URI/SCRAPY_LOG_FILE if items_dir/logs_dir settings are not set
2012-05-08 17:43:00 -03:00
lostsnow
5afe4f50c1
scrapyd: support bind to a specific ip address
2012-02-29 13:47:40 +08:00
Pablo Hoffman
0b0bce7f3c
scrapyd: added cancel.json and listjobs.json api methods to documentation
2012-01-05 11:23:25 -02:00
Pablo Hoffman
8f42633a94
scrapyd: added clarification about how to disable items feeds generation
2012-01-05 11:20:50 -02:00
Pablo Hoffman
dbda33efa6
scrapyd: added support for storing items by default
...
Items are stored the same way as logs, in jsonlines format.
Also renamed logs_to_keep setting to jobs_to_keep.
2012-01-03 23:08:54 -02:00
Pablo Hoffman
0eeff76227
fixed formatting of scrapyd doc
2011-12-20 03:18:37 -02:00
Pablo Hoffman
8bdf288428
made scrapyd doc more version agnostic
2011-10-23 05:29:54 -02:00
Pablo Hoffman
431441cb52
updated documentation to remove references to old issue tracker and mercurial repos
2011-09-25 13:06:24 -03:00
Julien Duponchelle
b7c436343a
scrapy deploy support git version
2011-09-21 22:17:08 +02:00
Pablo Hoffman
bff3d31469
scrapyd: updated schedule.json response format
2011-09-04 09:29:24 -03:00
Pablo Hoffman
98b68ca89d
scrapyd: documented support for passing setting to spiders in schedule.json
2011-08-27 01:31:12 -03:00
Pablo Hoffman
ad496eb3b6
Automated merge with ssh://hg.scrapy.org:2222/scrapy-0.12
2011-04-14 12:36:27 -03:00
Pablo Hoffman
ecb4f44cbc
Added clarification on how to work with local settings and scrapy deploy
2011-04-14 12:36:09 -03:00
Pablo Hoffman
d7f193cbea
bumped version to 0.13 in documentation
2011-01-02 17:29:43 -02:00
Pablo Hoffman
b56e933be9
bumped version to 0.12 in documentation
2011-01-02 17:28:33 -02:00
Pablo Hoffman
fa644f7a5e
Some simplifications to Scrapyd architecture and internals:
...
- launcher no longer knows about egg storage
- removed get_spider_list_from_eggifile() file and replaced by simpler
get_spider_list() which doesn't receive en egg file as argument
- changed "egg runner" name to just "runner" to reflect the fact that it
doesn't necesarilly run eggs (though it does in the default case)
--HG--
rename : scrapyd/eggrunner.py => scrapyd/runner.py
2010-12-27 16:22:32 -02:00
Pablo Hoffman
f984d438a0
updated docs to use scrapy version on aptitude install lines
2010-12-13 14:02:42 -02:00
Pablo Hoffman
6a1b69c93f
renamed command 'scrapyd' to 'server', and deprecated 'runserver' and 'queue' commands
...
--HG--
rename : scrapy/commands/scrapyd.py => scrapy/commands/server.py
2010-11-30 20:23:27 -02:00
Pablo Hoffman
df54ed0041
Some Scrapyd enhancements:
...
* added minimal web ui
* return unique id per job (spider scheduled)
* store one log per spider run (job) and rotate them, keeping the last N logs (where N is configurable through settings)
2010-11-30 02:26:31 -02:00
Pablo Hoffman
bbffa59497
Some changes to Scrapyd:
...
* Always start one process per spider
* Added max_proc_per_cpu option (defaults to 4)
* Return the number of spiders (instead of a list of them) in schedule.json
2010-11-29 17:19:05 -02:00
Pablo Hoffman
d988ca1ec2
Some changes to scrapy deploy command:
...
* changed deploy section names to [deploy:target]
* project is now passed through a -p|--project option
* version can now be set in the target configuration
* switched meaning of -l and -L options
* updated documentation accordingly
2010-11-08 17:01:06 -02:00
Pablo Hoffman
dfa6745e91
Automated merge with http://hg.scrapy.org/scrapy-0.10
2010-10-30 16:05:53 -02:00
Pablo Hoffman
a0d9b43031
fixed typo in scrapyd doc
2010-10-30 16:05:32 -02:00
Pablo Hoffman
d67152ab0f
Automated merge with http://hg.scrapy.org/scrapy-0.10
2010-10-30 01:56:12 -02:00
Pablo Hoffman
75451cbe84
scrapyd doc: fixed delversion.json example
2010-10-30 01:56:00 -02:00
Pablo Hoffman
6c921896a5
Expanded documentation on deploy command and versions. Refs #261
2010-10-19 00:11:45 -02:00
Pablo Hoffman
1d567cdce6
Added new 'deploy' command. Closes #261
2010-10-18 22:38:46 -02:00
Pablo Hoffman
98662e53ea
Formatting fix in Scrapyd doc
2010-10-17 03:20:23 -02:00
Pablo Hoffman
9158e9d682
Some changes to Scrapyd to support multiple configuration files, to make it easier to deploy Scrapyd applications. Also documented 'egg_runner' and 'application' options
...
--HG--
rename : debian/scrapyd.cfg => debian/000-default
rename : scrapyd/default_scrapyd.cfg => scrapyd/default_scrapyd.conf
2010-09-07 09:17:25 -03:00
Pablo Hoffman
bf34094e5a
Added versionadded:: notice to new documentation topics
2010-09-04 03:30:45 -03:00
Pablo Hoffman
37e9c5d78e
Added new Scrapy service with support for:
...
* multiple projects
* uploading scrapy projects as Python eggs
* scheduling spiders using a JSON API
Documentation is added along with the code.
Closes #218 .
--HG--
rename : debian/scrapy-service.default => debian/scrapyd.default
rename : debian/scrapy-service.dirs => debian/scrapyd.dirs
rename : debian/scrapy-service.install => debian/scrapyd.install
rename : debian/scrapy-service.lintian-overrides => debian/scrapyd.lintian-overrides
rename : debian/scrapy-service.postinst => debian/scrapyd.postinst
rename : debian/scrapy-service.postrm => debian/scrapyd.postrm
rename : debian/scrapy-service.upstart => debian/scrapyd.upstart
rename : extras/scrapy.tac => extras/scrapyd.tac
2010-09-03 15:54:42 -03:00
