1
0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-28 13:04:05 +00:00

218 Commits

Author SHA1 Message Date
Pablo Hoffman
a0eec7eaf6 some typos fixes and updates to install doc 2009-09-29 09:44:02 -03:00
Ismael Carnales
1646482bef reformatted installation guide 2009-09-29 08:41:34 -03:00
Pablo Hoffman
4a9d9282bc removed obsolete scrapy architecture dia diagram 2009-09-28 23:54:01 -03:00
Ismael Carnales
5862ba7db7 modified doc to reflect the new spider callback return policy (lists not needed) 2009-09-22 11:25:40 -03:00
Ismael Carnales
802f918b69 removed obsolete doc static file 2009-09-22 11:25:38 -03:00
Pablo Hoffman
6e93872955 updated installation guide for using releases 2009-09-17 11:06:55 -03:00
Pablo Hoffman
132557dd14 some deployment changes in preparation for the 0.7.0 release candidate 2009-09-16 22:40:36 -03:00
Ismael Carnales
fd41f06056 added doc on how to enable an Item Pipeline component 2009-09-16 14:19:16 -03:00
Ismael Carnales
404e7e09d7 changed spider doc references in BaseSpider class 2009-09-16 14:10:11 -03:00
Daniel Grana
062730cbd8 fix csv exporter documentation 2009-09-16 00:17:50 -03:00
Pablo Hoffman
56b292e057 XmlItemExporter: added built-in support for exporting multi-valued fields (for convenience) 2009-09-14 22:05:52 -03:00
Pablo Hoffman
e8960bf616 added runspider command to run spiders directly, without having to create a project 2009-09-14 22:05:14 -03:00
Pablo Hoffman
99467d4e6e Changed (unstable) scheduler middleware API to receive spider (instead of domain) in enqueue_request method 2009-09-13 20:51:43 -03:00
Pablo Hoffman
921fc4f3bf Big Scrapy core refactoring to pass around spider references instead of domains.
This is to avoid accessing the scrapy.spider.spiders singleton for "resolving"
spiders, which is considered an "evil" practice because it ties us to the
singleton model for the spider resolver, which is a bad thing.

This change will also work as the foundation for the API cleaning that we'll
perform for 0.8. We decided to introduce this change now to have a more common
basecode between 0.7 and 0.8, which will allow us to better support 0.7 until
0.8 is released.

However, this change doesn't modify the stable/documented API, nor does it
change the core logic. Those changes will land on the 0.8 branch, after 0.7 is
released.

--HG--
rename : scrapy/contrib/domainsch.py => scrapy/contrib/spiderscheduler.py
2009-09-12 14:34:18 -03:00
Pablo Hoffman
8d49dc2fb5 changed IMAGES_THUMBS setting to a dict instead of a list of tuples, and more improvements to images pipeline doc 2009-09-11 17:36:00 -03:00
Pablo Hoffman
e20f766792 fixed some typos 2009-09-11 16:55:37 -03:00
Pablo Hoffman
c2fe350f72 more changes to images pipeline doc 2009-09-11 16:53:36 -03:00
Ismael Carnales
ada46a2dbb styled imagesp doc 2009-09-11 15:30:46 -03:00
Pablo Hoffman
be0f2beef0 more cleanup to scheduler middelware doc, and permanentely moved to experimental doc 2009-09-11 13:27:31 -03:00
Pablo Hoffman
0af052b68f removed confusing title 2009-09-11 12:19:18 -03:00
Pablo Hoffman
f3240748cb changed link to scheduler middleware doc, now in experimental 2009-09-11 12:03:23 -03:00
Ismael Carnales
3998a0cb58 added more scheduler middleware documentation, and moved it to experimental
--HG--
rename : docs/topics/scheduler-middleware.rst => docs/experimental/scheduler-middleware.rst
2009-09-11 11:58:53 -03:00
Pablo Hoffman
d242a20573 updated images pipeline doc 2009-09-11 11:47:12 -03:00
Pablo Hoffman
f1bb8dc2a3 first cleanup of spider manager api
- removed asdict() and reload() methods
- added list() method
- removed default spider
2009-09-10 19:06:46 -03:00
Pablo Hoffman
f85813cd94 added FAQ entry about scrapy recipes and community spiders 2009-09-10 18:32:50 -03:00
Pablo Hoffman
269724a2b7 added Debugger extension, removed StackTraceDump from extensions available by default 2009-09-08 22:32:17 -03:00
Ismael Carnales
4ddfa9a2a3 stlyed downloaded middleware doc 2009-09-07 12:18:57 -03:00
Ismael Carnales
e3df11e5bb added module directive to spidermw documentation 2009-09-07 12:03:24 -03:00
Pablo Hoffman
827aa19c6e removed obsolete scrapy.utils.db module 2009-09-04 17:38:14 -03:00
Pablo Hoffman
861a803cc3 removed obsolete RestrictMiddleware 2009-09-04 17:22:56 -03:00
Ismael Carnales
7e2587169b added missing middleware docs 2009-09-04 12:39:02 -03:00
Pablo Hoffman
aefb94063a more updates to spider middleware doc 2009-09-04 13:46:04 -03:00
Pablo Hoffman
d04640be5c some improvements to spider middleware doc 2009-09-04 13:29:16 -03:00
Pablo Hoffman
96bb223c13 removed (pretty useless) DebugMiddleware 2009-09-04 12:59:58 -03:00
Pablo Hoffman
8a715701ec fixed another doc typo 2009-09-03 14:31:00 -03:00
Ismael Carnales
3c1bb7bc40 fixed typo in djangoitems doc (thanks anibal) 2009-09-03 11:23:25 -03:00
Daniel Grana
0e7b2a6da5 write header line by default when using csv exporter
--HG--
extra : rebase_source : 2d2d7153dde5e3f77e682e16d2e4408f732f234e
2009-09-03 13:58:39 -03:00
Pablo Hoffman
596d2c4479 moved CoreStats extension to scrapy.contrib.corestats
--HG--
rename : scrapy/stats/corestats.py => scrapy/contrib/corestats.py
2009-09-01 23:00:49 -03:00
Pablo Hoffman
6a50af05d7 removed useless SpiderReloader extension 2009-09-01 22:49:15 -03:00
Pablo Hoffman
79851aefa6 moved SpiderProfiler extension to scrapy.contrib_exp and removed references from documentation
--HG--
rename : scrapy/contrib/spider/profiler.py => scrapy/contrib_exp/spiderprofiler.py
2009-09-01 22:38:37 -03:00
Pablo Hoffman
d3c51fd6f2 improved images pipeline documentation 2009-09-01 21:07:47 -03:00
Pablo Hoffman
18fd635124 another doc typo 2009-09-01 12:52:40 -03:00
Pablo Hoffman
538cc9803a fixed doc typo 2009-09-01 12:47:53 -03:00
Pablo Hoffman
df0e1f005f exporters doc: fixed example and some typos 2009-09-01 08:56:54 -03:00
Pablo Hoffman
ac8f46ce9e added File Export Pipeline reference to Exporters doc 2009-08-31 21:01:35 -03:00
Pablo Hoffman
8d006e9ea1 moved item exporters doc to stable doc
--HG--
rename : docs/experimental/exporters.rst => docs/topics/exporters.rst
2009-08-31 20:47:12 -03:00
Pablo Hoffman
0b152c99b5 added File Export Pipeline, a wrapper to use Item Exporters as Item Pipelines 2009-08-31 20:40:41 -03:00
Pablo Hoffman
8fab524978 moved engine.getstatus() method to scrapy.utils.engine function, to leave reporting logic out of engine code. added est() shortcut to telnet console 2009-08-31 12:44:32 -03:00
Pablo Hoffman
884f0c878f Stats collectin: fixed race condition between stats persistance and population of stats on domain close 2009-08-29 19:44:13 -03:00
Pablo Hoffman
895c70e036 doc: fixed some links to scrapy-ctl topic 2009-08-29 18:23:55 -03:00