1524 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Pablo Hoffman	b763350340	removed unused module: scrapy.contrib_exp.history	2009-08-20 14:29:39 -03:00
Pablo Hoffman	b65c87cb67	removed unused module: scrapy.utils.c14n	2009-08-20 14:23:49 -03:00
Pablo Hoffman	5de409a6a0	removed unused module: scrapy.tests.serialization	2009-08-20 14:20:45 -03:00
Pablo Hoffman	20e82335e2	rename some exporter methods and complete exporter tests refactoring	2009-08-20 14:09:24 -03:00
Ismael Carnales	6b7162f3ca	updated JsonLinesItemExporter to new exporters API	2009-08-20 12:58:10 -03:00
Pablo Hoffman	3152e66fec	deprecate domain_open signal and handle stats domain open/close directly from the engine	2009-08-20 10:54:58 -03:00
Pablo Hoffman	5522560d36	updated example project to use new selectors module	2009-08-20 10:25:43 -03:00
Pablo Hoffman	06db0bc124	updated some documentation references in source code	2009-08-19 22:41:26 -03:00
Pablo Hoffman	33b53c59d5	moved scrapy.xpath to scrapy.selector ... --HG-- rename : scrapy/xpath/__init__.py => scrapy/selector/__init__.py rename : scrapy/xpath/document.py => scrapy/selector/document.py rename : scrapy/xpath/factories.py => scrapy/selector/factories.py	2009-08-19 21:50:52 -03:00
Pablo Hoffman	dcc90fc196	declared loaders api stable and updated example project to use them ... --HG-- rename : docs/experimental/loaders.rst => docs/topics/loaders.rst	2009-08-19 21:39:59 -03:00
Pablo Hoffman	e8504a054c	moved scrapy.newitem to scrapy.item and declared newitem api officially stable. updated docs and example project. deprecated old ScrapedItem	2009-08-19 21:39:58 -03:00
Ismael Carnales	314e8dea43	added new item exporter tests, introduced some api changes	2009-08-19 19:05:33 -03:00
Pablo Hoffman	907fad6da8	make sure input processors always receive iterables as input	2009-08-19 16:49:49 -03:00
Pablo Hoffman	1d351c2866	minor change to offsite middleware regex, for clarity (doesn't change behaviour)	2009-08-19 16:16:34 -03:00
Pablo Hoffman	f3c6d83ad3	added check to CsvItemExporter	2009-08-19 15:20:29 -03:00
Pablo Hoffman	741a6d7863	item exporters refactoring ... --HG-- rename : scrapy/contrib/exporter/jsonexporter.py => scrapy/contrib/exporter/jsonlines.py	2009-08-19 13:09:39 -03:00
Pablo Hoffman	9c5dad3f89	renamed scrapy.utils.ref module to scrapy.utils.trackref, and improved docstring ... --HG-- rename : scrapy/utils/ref.py => scrapy/utils/trackref.py	2009-08-19 11:19:36 -03:00
Pablo Hoffman	e947e1d45b	added some unittests to make sure certain objects are using __slots__ and are also weak-referenceable	2009-08-18 20:40:04 -03:00
Pablo Hoffman	240e06813d	scarpy.xpath: added __weakref__ to __slots__, removed unused XPathSelector.response attribute, moved deprecated methods to the bottom	2009-08-18 20:00:54 -03:00
Pablo Hoffman	00527fc93c	added __slots__ to XPathSelector and Libxml2Document classes	2009-08-18 19:57:05 -03:00
Pablo Hoffman	64ffe6b2d3	added scrapy.utils.ref module for tracking references to live instances, for certain objects	2009-08-18 19:44:25 -03:00
Pablo Hoffman	82e4b6adcf	merge with ismael repo	2009-08-18 15:38:20 -03:00
Pablo Hoffman	e01d31c498	another improvement to doc navbar	2009-08-18 15:35:53 -03:00
Ismael Carnales	be4226cec5	merge	2009-08-18 15:21:39 -03:00
Ismael Carnales	f88fb27851	fixed error in xpath selectors doc	2009-08-18 15:18:49 -03:00
Ismael Carnales	67c0c6a9e4	corrected indentation in xpath selectors doc	2009-08-18 15:13:23 -03:00
Pablo Hoffman	ff837e5a45	doc: improved top navbar	2009-08-18 15:12:44 -03:00
Ismael Carnales	428dfe0d4a	corrected the style of spiders documentation	2009-08-18 15:06:33 -03:00
Pablo Hoffman	0192282d07	reorganized doc and moved robotstxt doc inside downloader middlewares doc	2009-08-18 14:36:18 -03:00
Ismael Carnales	33089d287d	merged topics and reference doc	2009-08-18 14:05:15 -03:00
Pablo Hoffman	e82d9d885e	some speedups to offsite spider middleware using regexes and urlparse_cached	2009-08-18 12:43:25 -03:00
Pablo Hoffman	7f30461410	added support for defining EXTENSIONS setting using dicts, like middleware settings ... --HG-- rename : scrapy/tests/test_utils_middleware.py => scrapy/tests/test_utils_conf.py rename : scrapy/utils/middleware.py => scrapy/utils/conf.py	2009-08-18 11:05:36 -03:00
Ismael Carnales	1bbe7991dc	added documentation for ImagesPipeline	2009-08-18 09:35:32 -03:00
Ismael Carnales	ec8f934a8e	corrected import path in scrapy-admin.py	2009-08-18 09:02:23 -03:00
Pablo Hoffman	4b910881da	make sure get_vmvalue_from_procfs returns int	2009-08-18 00:59:32 -03:00
Daniel Grana	49cc1b1e7e	add missing future import for python 2.5	2009-08-17 21:42:37 -03:00
Pablo Hoffman	16c51353f2	updated select() method in crawl spider template	2009-08-17 21:22:05 -03:00
Pablo Hoffman	9ae3a1946d	remove Url class and use str instead for Request and Response urls. Also added urlparse_cached function for achieving the same caching functionality provided by old Url class	2009-08-17 21:16:55 -03:00
Pablo Hoffman	29710461f6	some refactoring to robotstxt downloader middleware	2009-08-17 19:11:43 -03:00
Pablo Hoffman	d6da7eb04f	removed backwards compatibility alias: load_class	2009-08-17 18:32:24 -03:00
Pablo Hoffman	e71764e707	removed unused functions: memoize, gzip_file	2009-08-17 18:30:40 -03:00
Pablo Hoffman	50de128e0d	removed unused items_to_csv function	2009-08-17 18:21:21 -03:00
Pablo Hoffman	6dfda357ef	removed unused dict_updatedefault function	2009-08-17 18:19:59 -03:00
Pablo Hoffman	c81987f624	removed unused hash_values function	2009-08-17 18:16:44 -03:00
Pablo Hoffman	4f06f6ef41	applied fix to deprecated decorator to warn only once (thanks Dan)	2009-08-17 18:13:45 -03:00
Pablo Hoffman	41036af643	some refactoring to genspider command	2009-08-17 17:59:34 -03:00
Ismael Carnales	48b40bd620	renamed x method of selectors to select	2009-08-17 15:58:06 -03:00
Pablo Hoffman	59e0a83ad4	removed more obsolete adaptors code	2009-08-17 14:48:11 -03:00
Pablo Hoffman	1bf5682144	removed unused modules	2009-08-17 14:41:06 -03:00
Pablo Hoffman	62c29e4d69	removed unused module	2009-08-17 14:39:54 -03:00