Daniel Grana
45cbe1333f
fix typo in stats docs
2009-08-11 16:30:04 -03:00
Daniel Grana
8f16e7f9fa
remove default parameter from max_value/min_value stats methods, update docs, and add new methods to dummy collector
2009-08-11 16:23:23 -03:00
Daniel Grana
cd8eed77c5
stats collector gains two new methods to store values only if greater/lower than previous value
2009-08-11 15:46:44 -03:00
Ismael Carnales
3614998053
fixed error in doc
2009-08-11 09:23:45 -03:00
Pablo Hoffman
7ac7597950
removed obsolete scrapy.contrib.cluster
2009-08-10 21:02:46 -03:00
Pablo Hoffman
c0532712f9
removed unnecesary response ResponseSoup extension, and replaced by a utils function with cache support
2009-08-10 21:02:24 -03:00
Pablo Hoffman
d334c035c0
removed unnecesary ResponseLibxml2 extension and moved libxml2 document caching functionality to Libxml2Document using weak references
2009-08-10 20:52:43 -03:00
Pablo Hoffman
49bab4777b
XPathSelector: added 're' argument to add_xpath method, exposed selector attribute
2009-08-10 19:42:20 -03:00
Pablo Hoffman
50fcc67278
improved reducers examples
2009-08-10 10:13:30 -03:00
Pablo Hoffman
5be0cee577
added TreeExpander example
2009-08-09 20:54:22 -03:00
Pablo Hoffman
38f82e3993
loaders doc: added information about expanders/reducers declaration precendece, and other minor improvements
2009-08-09 18:06:12 -03:00
Pablo Hoffman
b296d4169e
minor doc update for making it more windows-friendly
2009-08-09 17:08:42 -03:00
Pablo Hoffman
91eea82eef
added XPathLoader for working with XPath Selectors more conveniently
2009-08-08 07:26:20 -03:00
Pablo Hoffman
db90e26a8b
renamed ItemLoader class to Loader
2009-08-07 14:39:30 -03:00
Pablo Hoffman
e585c6cac4
relocated experimental newitems/loaders doc, and added example for extending fields metadata
...
--HG--
rename : docs/experimental/newitem-loader.rst => docs/experimental/loaders.rst
rename : docs/experimental/newitem.rst => docs/experimental/newitems.rst
2009-08-07 14:28:58 -03:00
Pablo Hoffman
d95e99f585
Added documentation for Items and Loaders, removed obsolete Item Adaptors documentation
...
--HG--
rename : docs/experimental/topics/newitem/index.rst => docs/experimental/newitem.rst
2009-08-07 03:50:09 -03:00
Ismael Carnales
32894643a0
added ListField documentation, ordered field reference alphabetically
2009-08-03 15:00:04 -03:00
Pablo Hoffman
c3427e075c
added domain_stats parameter to stats_domain_closed signal
2009-07-31 16:36:35 -03:00
Pablo Hoffman
73172b244d
added from_unicode_list() method to Field objects
2009-07-30 16:58:24 -03:00
Pablo Hoffman
8ec7c9e01c
WEBCONSOLE_PORT setting now defaults to 6080
2009-07-29 18:59:34 -03:00
Pablo Hoffman
fcc91901eb
finished cleaning up closedomain documentation, and updated default settings
2009-07-27 15:42:35 -03:00
Pablo Hoffman
09ba6927d7
Some changes to CloseDomain extension:
...
- added support for closing by item passed count (CLOSEDOMAIN_ITEMPASSED)
- removed support for sending notification emails (since that's the job of
another extension)
2009-07-27 15:23:50 -03:00
Pablo Hoffman
9da66698f3
moved httprepr() method (from Request and Response objects) to scrapy.utils functions
2009-07-25 18:56:12 -03:00
Pablo Hoffman
c615ace6bd
doc: updated google directory links in firebug guide
2009-07-24 13:14:36 -03:00
Daniel Grana
96c3cdbec2
improve OffsiteMiddleware reference docs
...
--HG--
extra : rebase_source : 3ed3f23fc1ec63b521ead029c5749898f3ab05d7
2009-07-24 12:59:38 -03:00
Pablo Hoffman
38c3f7d0b4
Some changes to logging of scraped items:
...
1. "Scraped Item" log level changed to DEBUG
2. "Dropped Item" log level changed to WARNING
3. added "Passed Item" log message with INFO level
2009-07-23 11:49:48 -03:00
Pablo Hoffman
e43e28bf1d
minimal doc improvement
2009-07-23 09:12:49 -03:00
Ismael Carnales
6d24ae5920
added reference to working with relative xpaths in the tutorial
2009-07-23 09:05:14 -03:00
Pablo Hoffman
9baa6bb2a8
minor selectors doc fix
2009-07-23 01:56:56 -03:00
Ismael Carnales
3c4afb23be
moved newitem from scrapy.contrib_exp to scrapy.newitem
2009-07-22 15:13:36 -03:00
Ismael Carnales
202894dd8f
fixes to newitem doc
2009-07-22 10:25:22 -03:00
Pablo Hoffman
7d8ba0542c
fixed typo in doc
2009-07-21 17:38:46 -03:00
Pablo Hoffman
aa345e116a
Added spider middleware documentation
2009-07-21 17:19:19 -03:00
Ismael Carnales
90f1d9e489
Added Scheduler middleware reference documentation
2009-07-21 16:52:27 -03:00
Ismael Carnales
00d9ce6608
fixes to newitem doc
2009-07-21 12:22:49 -03:00
Ismael Carnales
894a5d81d3
changed the newitem API to a dict-like interface
2009-07-21 12:20:49 -03:00
Pablo Hoffman
9867e7b2d6
removed references to obsolete ENABLED_SPIDERS_FILE setting
2009-07-21 01:11:48 -03:00
Pablo Hoffman
92b746d866
SimpledbStatsCollector: moved domain creation to constructor
2009-07-17 12:49:53 -03:00
Pablo Hoffman
292757f312
added link to architecture overview and fixed old link
2009-07-16 19:15:19 -03:00
Pablo Hoffman
136014d718
Added section about relative xpaths to XPathSelectors doc
2009-07-16 17:29:29 -03:00
Pablo Hoffman
0fd603e27e
Automated merge with http://hg.scrapy.org/users/ismael/scrapy-newitem/
2009-07-16 11:16:46 -03:00
Pablo Hoffman
c2f12327d6
added dumping of global scrapy stats at engine shutdown
2009-07-16 10:48:10 -03:00
Ismael Carnales
a9069e8eaa
reorganized experimental doc in topics and ref, slitted long documents, introduced minor changes
2009-07-16 09:58:44 -03:00
Pablo Hoffman
1125825996
doc: minor updates to tutorial
2009-07-15 22:10:00 -03:00
Pablo Hoffman
5582305648
renamed old STATS_DEBUG setting to STATS_DUMP
2009-07-15 21:02:51 -03:00
Pablo Hoffman
7f281cf295
some minor updates to doc
2009-07-15 01:40:44 -03:00
Pablo Hoffman
8480f49cf6
Complete stats refactoring and documentation
...
--HG--
rename : scrapy/stats/statscollector.py => scrapy/stats/collector/__init__.py
2009-07-15 00:30:40 -03:00
Ismael Carnales
0227c53b13
reorganize newitem documentation
2009-07-14 15:55:52 -03:00
Pablo Hoffman
7fb889349e
fixed link to experimental doc
2009-07-14 08:46:15 -03:00
Pablo Hoffman
f1dc4e4af2
corrected TextField examples
2009-07-14 08:39:38 -03:00
