Pablo Hoffman
a432c1ee40
updated logging doc to include new spider argument in log functions
2009-11-04 14:49:24 -02:00
Pablo Hoffman
97c322707a
* Renamed domain_{opened,closed,idle} signals to spider_{opened,closed,idle}
...
* Changed them to pass spider instances only (no domains) (refs #105 )
2009-11-03 00:39:02 -02:00
Pablo Hoffman
904cde6513
added clarification about new dont_click argument of FormRequest.from_response() method
2009-10-29 13:47:10 -02:00
Ismael Carnales
a244d23b89
added dont_click attr to FormRequest
2009-10-29 13:18:13 -02:00
Pablo Hoffman
b41c5b5d5b
fixed typo in intro/install doc (thanks phaithful)
2009-10-29 10:41:20 -02:00
Pablo Hoffman
9b5fef4f48
fixed typo in intro/install doc (thanks phaithful)
2009-10-28 09:34:31 -02:00
Pablo Hoffman
7296a7b889
added DEFAULT_RESPONSE_ENCODING setting
2009-10-21 16:13:41 -02:00
Pablo Hoffman
720bc166cf
updated new clickdata argument doc
2009-10-20 17:21:56 -02:00
Daniel Grana
6abb3c17ee
Improve FormRequest.from_response method to pass click data arguments to ClientForm library
2009-10-20 15:51:41 -02:00
Pablo Hoffman
2712d55cb9
Automated merge with http://hg.scrapy.org/scrapy-stable
2009-10-07 23:58:38 -02:00
Pablo Hoffman
bd481751d8
moved images pipeline documentation to stable doc
...
--HG--
rename : docs/experimental/images.rst => docs/topics/images.rst
2009-10-07 22:57:25 -02:00
Pablo Hoffman
b4d202a6b0
added note about memory usage extension not working on windows
2009-10-07 22:57:10 -02:00
Pablo Hoffman
937acd91d1
improved documentation of http proxy middleware
2009-10-07 21:00:34 -02:00
Pablo Hoffman
d03bf1f27f
Automated merge with http://hg.scrapy.org/scrapy-stable
2009-10-07 20:38:46 -02:00
Pablo Hoffman
37d9e015bb
minor fix to tutorial
2009-10-07 20:15:49 -02:00
Daniel Grana
bc64ca3e13
Add support to set http proxies per request, and obey enviroment variables http_proxy and no_proxy by default.
2009-10-05 04:10:22 -02:00
Daniel Grana
8aa7d153ae
rewrote of downloader handlers
...
* add REQUEST_HANDLERS setting with defaults for file, http and https schemes
* add documentation of new setting
* add unittests for all the builtin handlers
* remove unused getPage function
2009-10-05 04:10:22 -02:00
Pablo Hoffman
a0eec7eaf6
some typos fixes and updates to install doc
2009-09-29 09:44:02 -03:00
Ismael Carnales
1646482bef
reformatted installation guide
2009-09-29 08:41:34 -03:00
Pablo Hoffman
4a9d9282bc
removed obsolete scrapy architecture dia diagram
2009-09-28 23:54:01 -03:00
Ismael Carnales
5862ba7db7
modified doc to reflect the new spider callback return policy (lists not needed)
2009-09-22 11:25:40 -03:00
Ismael Carnales
802f918b69
removed obsolete doc static file
2009-09-22 11:25:38 -03:00
Pablo Hoffman
6e93872955
updated installation guide for using releases
2009-09-17 11:06:55 -03:00
Pablo Hoffman
132557dd14
some deployment changes in preparation for the 0.7.0 release candidate
2009-09-16 22:40:36 -03:00
Ismael Carnales
fd41f06056
added doc on how to enable an Item Pipeline component
2009-09-16 14:19:16 -03:00
Ismael Carnales
404e7e09d7
changed spider doc references in BaseSpider class
2009-09-16 14:10:11 -03:00
Daniel Grana
062730cbd8
fix csv exporter documentation
2009-09-16 00:17:50 -03:00
Pablo Hoffman
56b292e057
XmlItemExporter: added built-in support for exporting multi-valued fields (for convenience)
2009-09-14 22:05:52 -03:00
Pablo Hoffman
e8960bf616
added runspider command to run spiders directly, without having to create a project
2009-09-14 22:05:14 -03:00
Pablo Hoffman
99467d4e6e
Changed (unstable) scheduler middleware API to receive spider (instead of domain) in enqueue_request method
2009-09-13 20:51:43 -03:00
Pablo Hoffman
921fc4f3bf
Big Scrapy core refactoring to pass around spider references instead of domains.
...
This is to avoid accessing the scrapy.spider.spiders singleton for "resolving"
spiders, which is considered an "evil" practice because it ties us to the
singleton model for the spider resolver, which is a bad thing.
This change will also work as the foundation for the API cleaning that we'll
perform for 0.8. We decided to introduce this change now to have a more common
basecode between 0.7 and 0.8, which will allow us to better support 0.7 until
0.8 is released.
However, this change doesn't modify the stable/documented API, nor does it
change the core logic. Those changes will land on the 0.8 branch, after 0.7 is
released.
--HG--
rename : scrapy/contrib/domainsch.py => scrapy/contrib/spiderscheduler.py
2009-09-12 14:34:18 -03:00
Pablo Hoffman
8d49dc2fb5
changed IMAGES_THUMBS setting to a dict instead of a list of tuples, and more improvements to images pipeline doc
2009-09-11 17:36:00 -03:00
Pablo Hoffman
e20f766792
fixed some typos
2009-09-11 16:55:37 -03:00
Pablo Hoffman
c2fe350f72
more changes to images pipeline doc
2009-09-11 16:53:36 -03:00
Ismael Carnales
ada46a2dbb
styled imagesp doc
2009-09-11 15:30:46 -03:00
Pablo Hoffman
be0f2beef0
more cleanup to scheduler middelware doc, and permanentely moved to experimental doc
2009-09-11 13:27:31 -03:00
Pablo Hoffman
0af052b68f
removed confusing title
2009-09-11 12:19:18 -03:00
Pablo Hoffman
f3240748cb
changed link to scheduler middleware doc, now in experimental
2009-09-11 12:03:23 -03:00
Ismael Carnales
3998a0cb58
added more scheduler middleware documentation, and moved it to experimental
...
--HG--
rename : docs/topics/scheduler-middleware.rst => docs/experimental/scheduler-middleware.rst
2009-09-11 11:58:53 -03:00
Pablo Hoffman
d242a20573
updated images pipeline doc
2009-09-11 11:47:12 -03:00
Pablo Hoffman
f1bb8dc2a3
first cleanup of spider manager api
...
- removed asdict() and reload() methods
- added list() method
- removed default spider
2009-09-10 19:06:46 -03:00
Pablo Hoffman
f85813cd94
added FAQ entry about scrapy recipes and community spiders
2009-09-10 18:32:50 -03:00
Pablo Hoffman
269724a2b7
added Debugger extension, removed StackTraceDump from extensions available by default
2009-09-08 22:32:17 -03:00
Ismael Carnales
4ddfa9a2a3
stlyed downloaded middleware doc
2009-09-07 12:18:57 -03:00
Ismael Carnales
e3df11e5bb
added module directive to spidermw documentation
2009-09-07 12:03:24 -03:00
Pablo Hoffman
827aa19c6e
removed obsolete scrapy.utils.db module
2009-09-04 17:38:14 -03:00
Pablo Hoffman
861a803cc3
removed obsolete RestrictMiddleware
2009-09-04 17:22:56 -03:00
Ismael Carnales
7e2587169b
added missing middleware docs
2009-09-04 12:39:02 -03:00
Pablo Hoffman
aefb94063a
more updates to spider middleware doc
2009-09-04 13:46:04 -03:00
Pablo Hoffman
d04640be5c
some improvements to spider middleware doc
2009-09-04 13:29:16 -03:00
