Ismael Carnales
fd41f06056
added doc on how to enable an Item Pipeline component
2009-09-16 14:19:16 -03:00
Ismael Carnales
404e7e09d7
changed spider doc references in BaseSpider class
2009-09-16 14:10:11 -03:00
Daniel Grana
062730cbd8
fix csv exporter documentation
2009-09-16 00:17:50 -03:00
Pablo Hoffman
56b292e057
XmlItemExporter: added built-in support for exporting multi-valued fields (for convenience)
2009-09-14 22:05:52 -03:00
Pablo Hoffman
921fc4f3bf
Big Scrapy core refactoring to pass around spider references instead of domains.
...
This is to avoid accessing the scrapy.spider.spiders singleton for "resolving"
spiders, which is considered an "evil" practice because it ties us to the
singleton model for the spider resolver, which is a bad thing.
This change will also work as the foundation for the API cleaning that we'll
perform for 0.8. We decided to introduce this change now to have a more common
basecode between 0.7 and 0.8, which will allow us to better support 0.7 until
0.8 is released.
However, this change doesn't modify the stable/documented API, nor does it
change the core logic. Those changes will land on the 0.8 branch, after 0.7 is
released.
--HG--
rename : scrapy/contrib/domainsch.py => scrapy/contrib/spiderscheduler.py
2009-09-12 14:34:18 -03:00
Ismael Carnales
3998a0cb58
added more scheduler middleware documentation, and moved it to experimental
...
--HG--
rename : docs/topics/scheduler-middleware.rst => docs/experimental/scheduler-middleware.rst
2009-09-11 11:58:53 -03:00
Pablo Hoffman
f1bb8dc2a3
first cleanup of spider manager api
...
- removed asdict() and reload() methods
- added list() method
- removed default spider
2009-09-10 19:06:46 -03:00
Pablo Hoffman
269724a2b7
added Debugger extension, removed StackTraceDump from extensions available by default
2009-09-08 22:32:17 -03:00
Ismael Carnales
4ddfa9a2a3
stlyed downloaded middleware doc
2009-09-07 12:18:57 -03:00
Ismael Carnales
e3df11e5bb
added module directive to spidermw documentation
2009-09-07 12:03:24 -03:00
Pablo Hoffman
827aa19c6e
removed obsolete scrapy.utils.db module
2009-09-04 17:38:14 -03:00
Pablo Hoffman
861a803cc3
removed obsolete RestrictMiddleware
2009-09-04 17:22:56 -03:00
Ismael Carnales
7e2587169b
added missing middleware docs
2009-09-04 12:39:02 -03:00
Pablo Hoffman
aefb94063a
more updates to spider middleware doc
2009-09-04 13:46:04 -03:00
Pablo Hoffman
d04640be5c
some improvements to spider middleware doc
2009-09-04 13:29:16 -03:00
Pablo Hoffman
96bb223c13
removed (pretty useless) DebugMiddleware
2009-09-04 12:59:58 -03:00
Daniel Grana
0e7b2a6da5
write header line by default when using csv exporter
...
--HG--
extra : rebase_source : 2d2d7153dde5e3f77e682e16d2e4408f732f234e
2009-09-03 13:58:39 -03:00
Pablo Hoffman
596d2c4479
moved CoreStats extension to scrapy.contrib.corestats
...
--HG--
rename : scrapy/stats/corestats.py => scrapy/contrib/corestats.py
2009-09-01 23:00:49 -03:00
Pablo Hoffman
6a50af05d7
removed useless SpiderReloader extension
2009-09-01 22:49:15 -03:00
Pablo Hoffman
79851aefa6
moved SpiderProfiler extension to scrapy.contrib_exp and removed references from documentation
...
--HG--
rename : scrapy/contrib/spider/profiler.py => scrapy/contrib_exp/spiderprofiler.py
2009-09-01 22:38:37 -03:00
Pablo Hoffman
18fd635124
another doc typo
2009-09-01 12:52:40 -03:00
Pablo Hoffman
538cc9803a
fixed doc typo
2009-09-01 12:47:53 -03:00
Pablo Hoffman
df0e1f005f
exporters doc: fixed example and some typos
2009-09-01 08:56:54 -03:00
Pablo Hoffman
ac8f46ce9e
added File Export Pipeline reference to Exporters doc
2009-08-31 21:01:35 -03:00
Pablo Hoffman
8d006e9ea1
moved item exporters doc to stable doc
...
--HG--
rename : docs/experimental/exporters.rst => docs/topics/exporters.rst
2009-08-31 20:47:12 -03:00
Pablo Hoffman
0b152c99b5
added File Export Pipeline, a wrapper to use Item Exporters as Item Pipelines
2009-08-31 20:40:41 -03:00
Pablo Hoffman
8fab524978
moved engine.getstatus() method to scrapy.utils.engine function, to leave reporting logic out of engine code. added est() shortcut to telnet console
2009-08-31 12:44:32 -03:00
Pablo Hoffman
884f0c878f
Stats collectin: fixed race condition between stats persistance and population of stats on domain close
2009-08-29 19:44:13 -03:00
Pablo Hoffman
895c70e036
doc: fixed some links to scrapy-ctl topic
2009-08-29 18:23:55 -03:00
Pablo Hoffman
64572124e0
added doc about SCRAPY_SETTINGS_MODULE
2009-08-29 18:20:13 -03:00
Pablo Hoffman
9dc7a3a373
some minor adjustments to logging doc
2009-08-29 18:10:28 -03:00
Pablo Hoffman
60cbf24c89
more cleanups to startproject and project templates
...
--HG--
rename : scrapy/templates/project/root/scrapy-ctl.py => scrapy/templates/project/scrapy-ctl.py
2009-08-29 04:29:47 -03:00
Pablo Hoffman
c87216f02b
doc: added missing :synopsis: to some modules
2009-08-29 03:46:58 -03:00
Pablo Hoffman
924219dd5a
- added reference documentation about scrapy-ctl.py script
...
- yet another refactor to cmdline module
- removed help command (use -h to help now)
--HG--
rename : docs/experimental/scripts.rst => docs/topics/scrapy-ctl.rst
2009-08-28 20:32:55 -03:00
Pablo Hoffman
609aed4553
added inspect_response() function for inspecting responses from spiders
2009-08-27 19:33:33 -03:00
Pablo Hoffman
ea4f16bae6
refactored scrapy shell implementation, dropping IPython dependency, and adding a new 'view' shortcut
2009-08-27 18:24:08 -03:00
Pablo Hoffman
65ab611fb0
more updates to HttpErrorMiddleware doc
2009-08-26 08:30:18 -03:00
Pablo Hoffman
4dba75a73f
updated HttpErrorMiddleware doc
2009-08-25 20:13:58 -03:00
Pablo Hoffman
8a074c9cb5
removed scrapy-admin.py command, and left only scrapy-ctl as the only scrapy command
2009-08-24 15:43:36 -03:00
Ismael Carnales
85282a4b76
added scrapy commandline scripts doc
2009-08-24 12:02:44 -03:00
Pablo Hoffman
ead8cb9fac
removed documentation about ugly DontCloseDomain exception (which will be removed in the future)
2009-08-24 11:57:21 -03:00
Pablo Hoffman
74f706b356
renamed "parse_item" method of XMLFeedSpider to "parse_node", keeping backwards compatibility
2009-08-24 10:54:34 -03:00
Pablo Hoffman
31693eb90f
dropped "cache" attribute of Request and Response objects
2009-08-24 10:34:05 -03:00
Pablo Hoffman
0186c6937a
HTTP auth middleware: added doc and unittest
2009-08-24 08:07:20 -03:00
Pablo Hoffman
46aaea9f32
doc: improved documentation about debugging leaks
2009-08-23 20:36:00 -03:00
Pablo Hoffman
9635a7839c
rearranged documentation into a better organization
...
--HG--
rename : docs/topics/index.rst => docs/index.rst
2009-08-21 21:49:54 -03:00
Pablo Hoffman
4761e0c88d
minor doc correction
2009-08-21 19:11:59 -03:00
Pablo Hoffman
ef6f04eb06
moved doc about debugging memory leaks to its own topic and added doc about trackref module
2009-08-21 16:07:16 -03:00
Pablo Hoffman
4980eb4968
added titles to signals doc
2009-08-21 15:07:52 -03:00
Pablo Hoffman
daea3f8a0c
sphinx docs: replaced custom :exception: xref by standard :exc:
2009-08-21 15:05:06 -03:00
