Pablo Hoffman
|
3152e66fec
|
deprecate domain_open signal and handle stats domain open/close directly from the engine
|
2009-08-20 10:54:58 -03:00 |
|
Pablo Hoffman
|
33b53c59d5
|
moved scrapy.xpath to scrapy.selector
--HG--
rename : scrapy/xpath/__init__.py => scrapy/selector/__init__.py
rename : scrapy/xpath/document.py => scrapy/selector/document.py
rename : scrapy/xpath/factories.py => scrapy/selector/factories.py
|
2009-08-19 21:50:52 -03:00 |
|
Pablo Hoffman
|
dcc90fc196
|
declared loaders api stable and updated example project to use them
--HG--
rename : docs/experimental/loaders.rst => docs/topics/loaders.rst
|
2009-08-19 21:39:59 -03:00 |
|
Pablo Hoffman
|
e8504a054c
|
moved scrapy.newitem to scrapy.item and declared newitem api officially stable. updated docs and example project. deprecated old ScrapedItem
|
2009-08-19 21:39:58 -03:00 |
|
Ismael Carnales
|
314e8dea43
|
added new item exporter tests, introduced some api changes
|
2009-08-19 19:05:33 -03:00 |
|
Pablo Hoffman
|
907fad6da8
|
make sure input processors always receive iterables as input
|
2009-08-19 16:49:49 -03:00 |
|
Pablo Hoffman
|
741a6d7863
|
item exporters refactoring
--HG--
rename : scrapy/contrib/exporter/jsonexporter.py => scrapy/contrib/exporter/jsonlines.py
|
2009-08-19 13:09:39 -03:00 |
|
Pablo Hoffman
|
82e4b6adcf
|
merge with ismael repo
|
2009-08-18 15:38:20 -03:00 |
|
Pablo Hoffman
|
e01d31c498
|
another improvement to doc navbar
|
2009-08-18 15:35:53 -03:00 |
|
Ismael Carnales
|
be4226cec5
|
merge
|
2009-08-18 15:21:39 -03:00 |
|
Ismael Carnales
|
f88fb27851
|
fixed error in xpath selectors doc
|
2009-08-18 15:18:49 -03:00 |
|
Ismael Carnales
|
67c0c6a9e4
|
corrected indentation in xpath selectors doc
|
2009-08-18 15:13:23 -03:00 |
|
Pablo Hoffman
|
ff837e5a45
|
doc: improved top navbar
|
2009-08-18 15:12:44 -03:00 |
|
Ismael Carnales
|
428dfe0d4a
|
corrected the style of spiders documentation
|
2009-08-18 15:06:33 -03:00 |
|
Pablo Hoffman
|
0192282d07
|
reorganized doc and moved robotstxt doc inside downloader middlewares doc
|
2009-08-18 14:36:18 -03:00 |
|
Ismael Carnales
|
33089d287d
|
merged topics and reference doc
|
2009-08-18 14:05:15 -03:00 |
|
Pablo Hoffman
|
7f30461410
|
added support for defining EXTENSIONS setting using dicts, like middleware settings
--HG--
rename : scrapy/tests/test_utils_middleware.py => scrapy/tests/test_utils_conf.py
rename : scrapy/utils/middleware.py => scrapy/utils/conf.py
|
2009-08-18 11:05:36 -03:00 |
|
Ismael Carnales
|
1bbe7991dc
|
added documentation for ImagesPipeline
|
2009-08-18 09:35:32 -03:00 |
|
Ismael Carnales
|
48b40bd620
|
renamed x method of selectors to select
|
2009-08-17 15:58:06 -03:00 |
|
Pablo Hoffman
|
59e0a83ad4
|
removed more obsolete adaptors code
|
2009-08-17 14:48:11 -03:00 |
|
Ismael Carnales
|
8c2f62ba9c
|
added Item Exporters documentation
|
2009-08-14 09:16:29 -03:00 |
|
Pablo Hoffman
|
02b01c3d2b
|
loaders doc: fixed outdated line
|
2009-08-13 23:24:24 -03:00 |
|
Daniel Grana
|
6a74b51371
|
remove dupes words in loaders doc, and unused import in example
--HG--
extra : rebase_source : ea7886725af7a54bd1031cb28f92efbdfe921d9e
|
2009-08-13 23:23:08 -03:00 |
|
Ismael Carnales
|
ecca7f6478
|
fixes to Item Loader doc
|
2009-08-13 13:30:50 -03:00 |
|
Pablo Hoffman
|
8131d674d4
|
fixed outdated documentation (refs #97)
|
2009-08-13 09:24:41 -03:00 |
|
Pablo Hoffman
|
5aeab5b291
|
converted scrapy.item package to module
--HG--
rename : scrapy/item/models.py => scrapy/item.py
|
2009-08-12 21:31:50 -03:00 |
|
Pablo Hoffman
|
991afdf6b3
|
some minor fixes to loaders doc
|
2009-08-12 19:23:04 -03:00 |
|
Pablo Hoffman
|
e6dd4d0955
|
renamed ApplyConcat processor to MapCompose
|
2009-08-12 18:43:08 -03:00 |
|
Pablo Hoffman
|
4e74f32401
|
renamed Pipe processor to Compose and documented it
|
2009-08-12 18:09:36 -03:00 |
|
Pablo Hoffman
|
958c3d9940
|
fixed some links to item loaders doc
|
2009-08-12 17:42:01 -03:00 |
|
Pablo Hoffman
|
d94b633565
|
renamed ItemLoader method populate_item() to load_item()
|
2009-08-12 17:40:27 -03:00 |
|
Pablo Hoffman
|
1dc592882b
|
Moved Item Loader to its final location in scrapy.contrib.loader, and updated doc/tests
--HG--
rename : docs/experimental/itemparser.rst => docs/experimental/loaders.rst
rename : scrapy/contrib/itemparser/__init__.py => scrapy/contrib/loader/__init__.py
rename : scrapy/contrib/itemparser/common.py => scrapy/contrib/loader/common.py
rename : scrapy/contrib/itemparser/parsers.py => scrapy/contrib/loader/processor.py
rename : scrapy/tests/test_itemparser.py => scrapy/tests/test_contrib_loader.py
|
2009-08-12 16:49:07 -03:00 |
|
Pablo Hoffman
|
7cbbc3ffb0
|
Renamed Loader to ItemParser (SEP-8 proposal). Documentation and unittests also updated.
--HG--
rename : docs/experimental/loaders.rst => docs/experimental/itemparser.rst
rename : scrapy/newitem/loader/__init__.py => scrapy/contrib/itemparser/__init__.py
rename : scrapy/newitem/loader/expanders.py => scrapy/contrib/itemparser/parsers.py
rename : scrapy/tests/test_itemloader.py => scrapy/tests/test_itemparser.py
|
2009-08-12 16:49:05 -03:00 |
|
Pablo Hoffman
|
d4aa72d2bb
|
removed obsolete documentation about Robust Scraped Item and Adaptors
|
2009-08-12 13:50:15 -03:00 |
|
Pablo Hoffman
|
e69311ef38
|
added missing text to new stats collector methods
|
2009-08-11 16:37:52 -03:00 |
|
Daniel Grana
|
45cbe1333f
|
fix typo in stats docs
|
2009-08-11 16:30:04 -03:00 |
|
Daniel Grana
|
8f16e7f9fa
|
remove default parameter from max_value/min_value stats methods, update docs, and add new methods to dummy collector
|
2009-08-11 16:23:23 -03:00 |
|
Daniel Grana
|
cd8eed77c5
|
stats collector gains two new methods to store values only if greater/lower than previous value
|
2009-08-11 15:46:44 -03:00 |
|
Ismael Carnales
|
3614998053
|
fixed error in doc
|
2009-08-11 09:23:45 -03:00 |
|
Pablo Hoffman
|
7ac7597950
|
removed obsolete scrapy.contrib.cluster
|
2009-08-10 21:02:46 -03:00 |
|
Pablo Hoffman
|
c0532712f9
|
removed unnecesary response ResponseSoup extension, and replaced by a utils function with cache support
|
2009-08-10 21:02:24 -03:00 |
|
Pablo Hoffman
|
d334c035c0
|
removed unnecesary ResponseLibxml2 extension and moved libxml2 document caching functionality to Libxml2Document using weak references
|
2009-08-10 20:52:43 -03:00 |
|
Pablo Hoffman
|
49bab4777b
|
XPathSelector: added 're' argument to add_xpath method, exposed selector attribute
|
2009-08-10 19:42:20 -03:00 |
|
Pablo Hoffman
|
50fcc67278
|
improved reducers examples
|
2009-08-10 10:13:30 -03:00 |
|
Pablo Hoffman
|
5be0cee577
|
added TreeExpander example
|
2009-08-09 20:54:22 -03:00 |
|
Pablo Hoffman
|
38f82e3993
|
loaders doc: added information about expanders/reducers declaration precendece, and other minor improvements
|
2009-08-09 18:06:12 -03:00 |
|
Pablo Hoffman
|
b296d4169e
|
minor doc update for making it more windows-friendly
|
2009-08-09 17:08:42 -03:00 |
|
Pablo Hoffman
|
91eea82eef
|
added XPathLoader for working with XPath Selectors more conveniently
|
2009-08-08 07:26:20 -03:00 |
|
Pablo Hoffman
|
db90e26a8b
|
renamed ItemLoader class to Loader
|
2009-08-07 14:39:30 -03:00 |
|
Pablo Hoffman
|
e585c6cac4
|
relocated experimental newitems/loaders doc, and added example for extending fields metadata
--HG--
rename : docs/experimental/newitem-loader.rst => docs/experimental/loaders.rst
rename : docs/experimental/newitem.rst => docs/experimental/newitems.rst
|
2009-08-07 14:28:58 -03:00 |
|