1904 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Daniel Grana	8ed743f265	remove obsolete s3 images pipeline	2009-09-02 15:38:53 -03:00
Pablo Hoffman	45bc12e268	added scrapy.utils.pdb module with set_trace() function	2009-09-02 12:09:36 -03:00
Daniel Grana	19c0093e50	fix images pipeline tests	2009-09-02 01:31:08 -03:00
Daniel Grana	11c820232b	remove MEDIA_NAME from imagespipeline ... --HG-- extra : rebase_source : 976f3ca2fccf185399a733904b9763498ebf43f6	2009-09-01 19:18:05 -03:00
Daniel Grana	d6adc141de	imagepipeline: simplify configuraiton using a single setting to setup different backend stores ... --HG-- extra : rebase_source : cd0d5f560dcde0003974e360282a233ba387c5be	2009-09-01 18:00:56 -03:00
Daniel Grana	b05fd80640	imagepipeline: return a dict instead of a path joined with checksum with a hash ... --HG-- extra : rebase_source : cd450c66be44381e905264ecff3f709bb9d302d4	2009-09-01 18:00:55 -03:00
Daniel Grana	91ff9c9d50	mediapipeline: simplify and return results of item_media_* to item_completed in order ... --HG-- extra : rebase_source : 71228e29b0a969df06ac62d90d6e3cefc03ecdfc	2009-09-01 18:00:55 -03:00
Pablo Hoffman	db5420f8d5	reload spider modules silently	2009-09-02 00:48:29 -03:00
Pablo Hoffman	596d2c4479	moved CoreStats extension to scrapy.contrib.corestats ... --HG-- rename : scrapy/stats/corestats.py => scrapy/contrib/corestats.py	2009-09-01 23:00:49 -03:00
Pablo Hoffman	5ce2b66c56	added proper recycling of spider resources to spider manager	2009-09-01 22:53:04 -03:00
Pablo Hoffman	34d559003d	removed empty scrapy.contrib.spider module	2009-09-01 22:50:36 -03:00
Pablo Hoffman	6a50af05d7	removed useless SpiderReloader extension	2009-09-01 22:49:15 -03:00
Pablo Hoffman	79851aefa6	moved SpiderProfiler extension to scrapy.contrib_exp and removed references from documentation ... --HG-- rename : scrapy/contrib/spider/profiler.py => scrapy/contrib_exp/spiderprofiler.py	2009-09-01 22:38:37 -03:00
Pablo Hoffman	d3c51fd6f2	improved images pipeline documentation	2009-09-01 21:07:47 -03:00
Pablo Hoffman	18fd635124	another doc typo	2009-09-01 12:52:40 -03:00
Pablo Hoffman	538cc9803a	fixed doc typo	2009-09-01 12:47:53 -03:00
Pablo Hoffman	e1b33ef165	added missing module from previous commit	2009-09-01 09:04:41 -03:00
Pablo Hoffman	df0e1f005f	exporters doc: fixed example and some typos	2009-09-01 08:56:54 -03:00
Pablo Hoffman	f68dedb284	commented out code that raises sporadically	2009-08-31 22:48:01 -03:00
Pablo Hoffman	ac8f46ce9e	added File Export Pipeline reference to Exporters doc	2009-08-31 21:01:35 -03:00
Pablo Hoffman	8d006e9ea1	moved item exporters doc to stable doc ... --HG-- rename : docs/experimental/exporters.rst => docs/topics/exporters.rst	2009-08-31 20:47:12 -03:00
Pablo Hoffman	0b152c99b5	added File Export Pipeline, a wrapper to use Item Exporters as Item Pipelines	2009-08-31 20:40:41 -03:00
Pablo Hoffman	3c6deaeffc	Some additional improvements to scrapy.command.cmdline logic: ... - calling scrapymanager.configure() for all commands - finally added some unittests to check cmdline behaviour!	2009-08-31 18:53:55 -03:00
Pablo Hoffman	6c58d06f0f	wrapped some big lines	2009-08-31 18:50:20 -03:00
Pablo Hoffman	b6c1392f9b	added 'settings' command for querying scrapy settings	2009-08-31 18:43:51 -03:00
Pablo Hoffman	60299c49ca	raise NotConfigured in web/telnetconsole when disabled	2009-08-31 13:42:44 -03:00
Pablo Hoffman	8fab524978	moved engine.getstatus() method to scrapy.utils.engine function, to leave reporting logic out of engine code. added est() shortcut to telnet console	2009-08-31 12:44:32 -03:00
Pablo Hoffman	dd80f6acdf	MemoryUsage: changed .virtual property to methodd. SpiderProfiler: removed dependency on MemoryUsage extension	2009-08-31 12:14:30 -03:00
Pablo Hoffman	e8e760c974	more simplifications to scrapy engine: removed addtasks method	2009-08-31 12:04:01 -03:00
Pablo Hoffman	1dd996f0b6	added missing import	2009-08-31 11:17:28 -03:00
Pablo Hoffman	8f013bf227	imported ismael patch for depreacting old SCRAPYSETTINGS_MODULE envvar	2009-08-31 10:17:12 -03:00
Pablo Hoffman	268acc30f5	merge with trunk	2009-08-31 09:44:55 -03:00
Pablo Hoffman	4f00c12a0a	removed unneeded logic from engine	2009-08-31 09:44:31 -03:00
Daniel Grana	91df6fcc7c	test genspider in a single testcase	2009-08-31 08:58:29 -03:00
Pablo Hoffman	bcf81cd3c4	minor simplification to how default settings are loaded	2009-08-31 07:36:09 -03:00
Pablo Hoffman	062761166c	simpledb collector: moved to_sdb_value function to utils.simpledb, and added unittests	2009-08-30 12:37:30 -03:00
Daniel Grana	1f50c24b6c	one minus command testing line	2009-08-30 01:55:39 -03:00
Daniel Grana	8ef2cf0add	more command testing simplifications	2009-08-30 01:51:34 -03:00
Daniel Grana	719fc2e7df	sdb stats: extend type serializations, allow timestamp to be other than now	2009-08-30 01:08:23 -03:00
Daniel Grana	7cb546ba28	use explicit relative import on djangoitem tests ... --HG-- extra : rebase_source : f4f72679106bd9f72ddd9ee7d1f27209248f59be	2009-08-29 21:04:32 -03:00
Pablo Hoffman	884f0c878f	Stats collectin: fixed race condition between stats persistance and population of stats on domain close	2009-08-29 19:44:13 -03:00
Pablo Hoffman	895c70e036	doc: fixed some links to scrapy-ctl topic	2009-08-29 18:23:55 -03:00
Pablo Hoffman	64572124e0	added doc about SCRAPY_SETTINGS_MODULE	2009-08-29 18:20:13 -03:00
Pablo Hoffman	9dc7a3a373	some minor adjustments to logging doc	2009-08-29 18:10:28 -03:00
Pablo Hoffman	91ee71222f	some minor changes to test_commands.py	2009-08-29 05:35:22 -03:00
Pablo Hoffman	5ba466a6e1	fixed unnapropiate handling of missing django module in djangitem tests	2009-08-29 05:03:40 -03:00
Pablo Hoffman	6e028fd91b	fixed JsonLinesItemExporterTest	2009-08-29 04:59:51 -03:00
Pablo Hoffman	60cbf24c89	more cleanups to startproject and project templates ... --HG-- rename : scrapy/templates/project/root/scrapy-ctl.py => scrapy/templates/project/scrapy-ctl.py	2009-08-29 04:29:47 -03:00
Pablo Hoffman	c87216f02b	doc: added missing :synopsis: to some modules	2009-08-29 03:46:58 -03:00
Pablo Hoffman	2f55d7932b	replaced :ref: by :doc: links in doc index	2009-08-29 03:37:59 -03:00