Daniel Grana
6a74b51371
remove dupes words in loaders doc, and unused import in example
...
--HG--
extra : rebase_source : ea7886725af7a54bd1031cb28f92efbdfe921d9e
2009-08-13 23:23:08 -03:00
Ismael Carnales
ecca7f6478
fixes to Item Loader doc
2009-08-13 13:30:50 -03:00
Pablo Hoffman
8131d674d4
fixed outdated documentation (refs #97 )
2009-08-13 09:24:41 -03:00
Pablo Hoffman
5aeab5b291
converted scrapy.item package to module
...
--HG--
rename : scrapy/item/models.py => scrapy/item.py
2009-08-12 21:31:50 -03:00
Pablo Hoffman
991afdf6b3
some minor fixes to loaders doc
2009-08-12 19:23:04 -03:00
Pablo Hoffman
e6dd4d0955
renamed ApplyConcat processor to MapCompose
2009-08-12 18:43:08 -03:00
Pablo Hoffman
4e74f32401
renamed Pipe processor to Compose and documented it
2009-08-12 18:09:36 -03:00
Pablo Hoffman
958c3d9940
fixed some links to item loaders doc
2009-08-12 17:42:01 -03:00
Pablo Hoffman
d94b633565
renamed ItemLoader method populate_item() to load_item()
2009-08-12 17:40:27 -03:00
Pablo Hoffman
1dc592882b
Moved Item Loader to its final location in scrapy.contrib.loader, and updated doc/tests
...
--HG--
rename : docs/experimental/itemparser.rst => docs/experimental/loaders.rst
rename : scrapy/contrib/itemparser/__init__.py => scrapy/contrib/loader/__init__.py
rename : scrapy/contrib/itemparser/common.py => scrapy/contrib/loader/common.py
rename : scrapy/contrib/itemparser/parsers.py => scrapy/contrib/loader/processor.py
rename : scrapy/tests/test_itemparser.py => scrapy/tests/test_contrib_loader.py
2009-08-12 16:49:07 -03:00
Pablo Hoffman
7cbbc3ffb0
Renamed Loader to ItemParser (SEP-8 proposal). Documentation and unittests also updated.
...
--HG--
rename : docs/experimental/loaders.rst => docs/experimental/itemparser.rst
rename : scrapy/newitem/loader/__init__.py => scrapy/contrib/itemparser/__init__.py
rename : scrapy/newitem/loader/expanders.py => scrapy/contrib/itemparser/parsers.py
rename : scrapy/tests/test_itemloader.py => scrapy/tests/test_itemparser.py
2009-08-12 16:49:05 -03:00
Pablo Hoffman
d4aa72d2bb
removed obsolete documentation about Robust Scraped Item and Adaptors
2009-08-12 13:50:15 -03:00
Pablo Hoffman
e69311ef38
added missing text to new stats collector methods
2009-08-11 16:37:52 -03:00
Daniel Grana
45cbe1333f
fix typo in stats docs
2009-08-11 16:30:04 -03:00
Daniel Grana
8f16e7f9fa
remove default parameter from max_value/min_value stats methods, update docs, and add new methods to dummy collector
2009-08-11 16:23:23 -03:00
Daniel Grana
cd8eed77c5
stats collector gains two new methods to store values only if greater/lower than previous value
2009-08-11 15:46:44 -03:00
Ismael Carnales
3614998053
fixed error in doc
2009-08-11 09:23:45 -03:00
Pablo Hoffman
7ac7597950
removed obsolete scrapy.contrib.cluster
2009-08-10 21:02:46 -03:00
Pablo Hoffman
c0532712f9
removed unnecesary response ResponseSoup extension, and replaced by a utils function with cache support
2009-08-10 21:02:24 -03:00
Pablo Hoffman
d334c035c0
removed unnecesary ResponseLibxml2 extension and moved libxml2 document caching functionality to Libxml2Document using weak references
2009-08-10 20:52:43 -03:00
Pablo Hoffman
49bab4777b
XPathSelector: added 're' argument to add_xpath method, exposed selector attribute
2009-08-10 19:42:20 -03:00
Pablo Hoffman
50fcc67278
improved reducers examples
2009-08-10 10:13:30 -03:00
Pablo Hoffman
5be0cee577
added TreeExpander example
2009-08-09 20:54:22 -03:00
Pablo Hoffman
38f82e3993
loaders doc: added information about expanders/reducers declaration precendece, and other minor improvements
2009-08-09 18:06:12 -03:00
Pablo Hoffman
b296d4169e
minor doc update for making it more windows-friendly
2009-08-09 17:08:42 -03:00
Pablo Hoffman
91eea82eef
added XPathLoader for working with XPath Selectors more conveniently
2009-08-08 07:26:20 -03:00
Pablo Hoffman
db90e26a8b
renamed ItemLoader class to Loader
2009-08-07 14:39:30 -03:00
Pablo Hoffman
e585c6cac4
relocated experimental newitems/loaders doc, and added example for extending fields metadata
...
--HG--
rename : docs/experimental/newitem-loader.rst => docs/experimental/loaders.rst
rename : docs/experimental/newitem.rst => docs/experimental/newitems.rst
2009-08-07 14:28:58 -03:00
Pablo Hoffman
d95e99f585
Added documentation for Items and Loaders, removed obsolete Item Adaptors documentation
...
--HG--
rename : docs/experimental/topics/newitem/index.rst => docs/experimental/newitem.rst
2009-08-07 03:50:09 -03:00
Ismael Carnales
32894643a0
added ListField documentation, ordered field reference alphabetically
2009-08-03 15:00:04 -03:00
Pablo Hoffman
c3427e075c
added domain_stats parameter to stats_domain_closed signal
2009-07-31 16:36:35 -03:00
Pablo Hoffman
73172b244d
added from_unicode_list() method to Field objects
2009-07-30 16:58:24 -03:00
Pablo Hoffman
8ec7c9e01c
WEBCONSOLE_PORT setting now defaults to 6080
2009-07-29 18:59:34 -03:00
Pablo Hoffman
fcc91901eb
finished cleaning up closedomain documentation, and updated default settings
2009-07-27 15:42:35 -03:00
Pablo Hoffman
09ba6927d7
Some changes to CloseDomain extension:
...
- added support for closing by item passed count (CLOSEDOMAIN_ITEMPASSED)
- removed support for sending notification emails (since that's the job of
another extension)
2009-07-27 15:23:50 -03:00
Pablo Hoffman
9da66698f3
moved httprepr() method (from Request and Response objects) to scrapy.utils functions
2009-07-25 18:56:12 -03:00
Pablo Hoffman
c615ace6bd
doc: updated google directory links in firebug guide
2009-07-24 13:14:36 -03:00
Daniel Grana
96c3cdbec2
improve OffsiteMiddleware reference docs
...
--HG--
extra : rebase_source : 3ed3f23fc1ec63b521ead029c5749898f3ab05d7
2009-07-24 12:59:38 -03:00
Pablo Hoffman
38c3f7d0b4
Some changes to logging of scraped items:
...
1. "Scraped Item" log level changed to DEBUG
2. "Dropped Item" log level changed to WARNING
3. added "Passed Item" log message with INFO level
2009-07-23 11:49:48 -03:00
Pablo Hoffman
e43e28bf1d
minimal doc improvement
2009-07-23 09:12:49 -03:00
Ismael Carnales
6d24ae5920
added reference to working with relative xpaths in the tutorial
2009-07-23 09:05:14 -03:00
Pablo Hoffman
9baa6bb2a8
minor selectors doc fix
2009-07-23 01:56:56 -03:00
Ismael Carnales
3c4afb23be
moved newitem from scrapy.contrib_exp to scrapy.newitem
2009-07-22 15:13:36 -03:00
Ismael Carnales
202894dd8f
fixes to newitem doc
2009-07-22 10:25:22 -03:00
Pablo Hoffman
7d8ba0542c
fixed typo in doc
2009-07-21 17:38:46 -03:00
Pablo Hoffman
aa345e116a
Added spider middleware documentation
2009-07-21 17:19:19 -03:00
Ismael Carnales
90f1d9e489
Added Scheduler middleware reference documentation
2009-07-21 16:52:27 -03:00
Ismael Carnales
00d9ce6608
fixes to newitem doc
2009-07-21 12:22:49 -03:00
Ismael Carnales
894a5d81d3
changed the newitem API to a dict-like interface
2009-07-21 12:20:49 -03:00
Pablo Hoffman
9867e7b2d6
removed references to obsolete ENABLED_SPIDERS_FILE setting
2009-07-21 01:11:48 -03:00
