Ismael Carnales
|
c08d3aa9cc
|
updated tutorial to use new items api
|
2009-08-21 14:16:27 -03:00 |
|
Pablo Hoffman
|
37760eb13f
|
improved consistency of logging settings to use LOG_*
|
2009-08-21 08:54:12 -03:00 |
|
Ismael Carnales
|
f0689a9ee8
|
fixed error in link extractors doc, thanks tarasm
|
2009-08-21 08:34:53 -03:00 |
|
Daniel Grana
|
bdc0ec6863
|
remove undefined variable from image pipeline
|
2009-08-20 20:30:08 -03:00 |
|
Pablo Hoffman
|
e7e1cba4a8
|
updated some docstings
|
2009-08-20 18:39:04 -03:00 |
|
Pablo Hoffman
|
41f1972e47
|
removed unused TRACE log level and improved logging documentation
|
2009-08-20 18:17:48 -03:00 |
|
Pablo Hoffman
|
297a8ac76b
|
moved caching resolver to an extension in contrib.resolver
|
2009-08-20 17:37:46 -03:00 |
|
Pablo Hoffman
|
a767b7044e
|
removed old blocking caching DNS resolver and replaced by a non-blocking one installed as the default reactor resolver
--HG--
rename : scrapy/core/downloader/dnscache.py => scrapy/core/downloader/resolver.py
|
2009-08-20 17:11:12 -03:00 |
|
Pablo Hoffman
|
cbed0f1ea6
|
moved send_catch_log to new scrapy.utils.signal module
|
2009-08-20 16:02:40 -03:00 |
|
Pablo Hoffman
|
8fad0a5744
|
fixed bug with defer_fail rename
|
2009-08-20 15:33:35 -03:00 |
|
Pablo Hoffman
|
eeac040fce
|
minor docstring update
|
2009-08-20 14:40:37 -03:00 |
|
Pablo Hoffman
|
119bb11090
|
removed unused chain_deferred function, renamed defer_fail to defer_failed
|
2009-08-20 14:37:38 -03:00 |
|
Pablo Hoffman
|
b763350340
|
removed unused module: scrapy.contrib_exp.history
|
2009-08-20 14:29:39 -03:00 |
|
Pablo Hoffman
|
b65c87cb67
|
removed unused module: scrapy.utils.c14n
|
2009-08-20 14:23:49 -03:00 |
|
Pablo Hoffman
|
5de409a6a0
|
removed unused module: scrapy.tests.serialization
|
2009-08-20 14:20:45 -03:00 |
|
Pablo Hoffman
|
20e82335e2
|
rename some exporter methods and complete exporter tests refactoring
|
2009-08-20 14:09:24 -03:00 |
|
Ismael Carnales
|
6b7162f3ca
|
updated JsonLinesItemExporter to new exporters API
|
2009-08-20 12:58:10 -03:00 |
|
Pablo Hoffman
|
3152e66fec
|
deprecate domain_open signal and handle stats domain open/close directly from the engine
|
2009-08-20 10:54:58 -03:00 |
|
Pablo Hoffman
|
5522560d36
|
updated example project to use new selectors module
|
2009-08-20 10:25:43 -03:00 |
|
Pablo Hoffman
|
06db0bc124
|
updated some documentation references in source code
|
2009-08-19 22:41:26 -03:00 |
|
Pablo Hoffman
|
33b53c59d5
|
moved scrapy.xpath to scrapy.selector
--HG--
rename : scrapy/xpath/__init__.py => scrapy/selector/__init__.py
rename : scrapy/xpath/document.py => scrapy/selector/document.py
rename : scrapy/xpath/factories.py => scrapy/selector/factories.py
|
2009-08-19 21:50:52 -03:00 |
|
Pablo Hoffman
|
dcc90fc196
|
declared loaders api stable and updated example project to use them
--HG--
rename : docs/experimental/loaders.rst => docs/topics/loaders.rst
|
2009-08-19 21:39:59 -03:00 |
|
Pablo Hoffman
|
e8504a054c
|
moved scrapy.newitem to scrapy.item and declared newitem api officially stable. updated docs and example project. deprecated old ScrapedItem
|
2009-08-19 21:39:58 -03:00 |
|
Ismael Carnales
|
314e8dea43
|
added new item exporter tests, introduced some api changes
|
2009-08-19 19:05:33 -03:00 |
|
Pablo Hoffman
|
907fad6da8
|
make sure input processors always receive iterables as input
|
2009-08-19 16:49:49 -03:00 |
|
Pablo Hoffman
|
1d351c2866
|
minor change to offsite middleware regex, for clarity (doesn't change behaviour)
|
2009-08-19 16:16:34 -03:00 |
|
Pablo Hoffman
|
f3c6d83ad3
|
added check to CsvItemExporter
|
2009-08-19 15:20:29 -03:00 |
|
Pablo Hoffman
|
741a6d7863
|
item exporters refactoring
--HG--
rename : scrapy/contrib/exporter/jsonexporter.py => scrapy/contrib/exporter/jsonlines.py
|
2009-08-19 13:09:39 -03:00 |
|
Pablo Hoffman
|
9c5dad3f89
|
renamed scrapy.utils.ref module to scrapy.utils.trackref, and improved docstring
--HG--
rename : scrapy/utils/ref.py => scrapy/utils/trackref.py
|
2009-08-19 11:19:36 -03:00 |
|
Pablo Hoffman
|
e947e1d45b
|
added some unittests to make sure certain objects are using __slots__ and are also weak-referenceable
|
2009-08-18 20:40:04 -03:00 |
|
Pablo Hoffman
|
240e06813d
|
scarpy.xpath: added __weakref__ to __slots__, removed unused XPathSelector.response attribute, moved deprecated methods to the bottom
|
2009-08-18 20:00:54 -03:00 |
|
Pablo Hoffman
|
00527fc93c
|
added __slots__ to XPathSelector and Libxml2Document classes
|
2009-08-18 19:57:05 -03:00 |
|
Pablo Hoffman
|
64ffe6b2d3
|
added scrapy.utils.ref module for tracking references to live instances, for certain objects
|
2009-08-18 19:44:25 -03:00 |
|
Pablo Hoffman
|
82e4b6adcf
|
merge with ismael repo
|
2009-08-18 15:38:20 -03:00 |
|
Pablo Hoffman
|
e01d31c498
|
another improvement to doc navbar
|
2009-08-18 15:35:53 -03:00 |
|
Ismael Carnales
|
be4226cec5
|
merge
|
2009-08-18 15:21:39 -03:00 |
|
Ismael Carnales
|
f88fb27851
|
fixed error in xpath selectors doc
|
2009-08-18 15:18:49 -03:00 |
|
Ismael Carnales
|
67c0c6a9e4
|
corrected indentation in xpath selectors doc
|
2009-08-18 15:13:23 -03:00 |
|
Pablo Hoffman
|
ff837e5a45
|
doc: improved top navbar
|
2009-08-18 15:12:44 -03:00 |
|
Ismael Carnales
|
428dfe0d4a
|
corrected the style of spiders documentation
|
2009-08-18 15:06:33 -03:00 |
|
Pablo Hoffman
|
0192282d07
|
reorganized doc and moved robotstxt doc inside downloader middlewares doc
|
2009-08-18 14:36:18 -03:00 |
|
Ismael Carnales
|
33089d287d
|
merged topics and reference doc
|
2009-08-18 14:05:15 -03:00 |
|
Pablo Hoffman
|
e82d9d885e
|
some speedups to offsite spider middleware using regexes and urlparse_cached
|
2009-08-18 12:43:25 -03:00 |
|
Pablo Hoffman
|
7f30461410
|
added support for defining EXTENSIONS setting using dicts, like middleware settings
--HG--
rename : scrapy/tests/test_utils_middleware.py => scrapy/tests/test_utils_conf.py
rename : scrapy/utils/middleware.py => scrapy/utils/conf.py
|
2009-08-18 11:05:36 -03:00 |
|
Ismael Carnales
|
1bbe7991dc
|
added documentation for ImagesPipeline
|
2009-08-18 09:35:32 -03:00 |
|
Ismael Carnales
|
ec8f934a8e
|
corrected import path in scrapy-admin.py
|
2009-08-18 09:02:23 -03:00 |
|
Pablo Hoffman
|
4b910881da
|
make sure get_vmvalue_from_procfs returns int
|
2009-08-18 00:59:32 -03:00 |
|
Daniel Grana
|
49cc1b1e7e
|
add missing future import for python 2.5
|
2009-08-17 21:42:37 -03:00 |
|
Pablo Hoffman
|
16c51353f2
|
updated select() method in crawl spider template
|
2009-08-17 21:22:05 -03:00 |
|
Pablo Hoffman
|
9ae3a1946d
|
remove Url class and use str instead for Request and Response urls. Also added urlparse_cached function for achieving the same caching functionality provided by old Url class
|
2009-08-17 21:16:55 -03:00 |
|