Pablo Hoffman
59acb129e5
scrapyd activate_egg(): don't override SCRAPY_SETTINGS_MODULE envvar if already set
2011-06-15 19:35:03 -03:00
Pablo Hoffman
fa644f7a5e
Some simplifications to Scrapyd architecture and internals:
...
- launcher no longer knows about egg storage
- removed get_spider_list_from_eggifile() file and replaced by simpler
get_spider_list() which doesn't receive en egg file as argument
- changed "egg runner" name to just "runner" to reflect the fact that it
doesn't necesarilly run eggs (though it does in the default case)
--HG--
rename : scrapyd/eggrunner.py => scrapyd/runner.py
2010-12-27 16:22:32 -02:00
Pablo Hoffman
5bdffadbe3
Simplified get_spider_list_from_eggfile() function now that it doesn't need to chdir to a custom directory (Scrapy now works when it's unable to create the SQLite database)
2010-11-05 11:48:12 -02:00
Pablo Hoffman
31bbcc9476
Raise error when egg is corrupt in activate_egg(). Use a more descriptive name for temporary dirs in get_spider_list_from_eggfile(). Make scrapyd webservice pass egg_runner to get_spider_list_from_eggfile()
2010-11-05 11:24:33 -02:00
Pablo Hoffman
de4909faca
get_spider_list_from_eggfile(): more improvements to error messages, and support passing eggruner module as argument
2010-11-04 18:56:11 -02:00
Pablo Hoffman
7ba972d8cf
get_spider_list_from_eggfile(): fail if unable to extract spider list
2010-11-04 16:27:47 -02:00
Pablo Hoffman
a4639ffb06
Removed hacky SCRAPY_SETTINGS_DISABLED environment variable
2010-09-22 16:08:18 -03:00
Pablo Hoffman
4c61df7abb
get_spider_list_from_eggfile(): fixed bug when SCRAPY_SETTINGS_DISABLED is set
2010-09-20 08:47:55 -03:00
Pablo Hoffman
400c4134af
Make scrapyd.eggutils compatible with Python 2.5 and added tests for get_spider_list_from_eggfile() function ( closes #242 )
2010-09-19 21:08:27 -03:00
Pablo Hoffman
833baa6041
Support running projects without eggs in Scrapyd. Closes #238
2010-09-14 01:44:25 -03:00
Pablo Hoffman
37e9c5d78e
Added new Scrapy service with support for:
...
* multiple projects
* uploading scrapy projects as Python eggs
* scheduling spiders using a JSON API
Documentation is added along with the code.
Closes #218 .
--HG--
rename : debian/scrapy-service.default => debian/scrapyd.default
rename : debian/scrapy-service.dirs => debian/scrapyd.dirs
rename : debian/scrapy-service.install => debian/scrapyd.install
rename : debian/scrapy-service.lintian-overrides => debian/scrapyd.lintian-overrides
rename : debian/scrapy-service.postinst => debian/scrapyd.postinst
rename : debian/scrapy-service.postrm => debian/scrapyd.postrm
rename : debian/scrapy-service.upstart => debian/scrapyd.upstart
rename : extras/scrapy.tac => extras/scrapyd.tac
2010-09-03 15:54:42 -03:00
