Pablo Hoffman
|
f565127eb2
|
added some missing file to MANIFEST.in
|
2009-09-28 23:55:00 -03:00 |
|
Pablo Hoffman
|
4a9d9282bc
|
removed obsolete scrapy architecture dia diagram
|
2009-09-28 23:54:01 -03:00 |
|
Pablo Hoffman
|
7e12d3d41b
|
removed old adaptors tests sample data
|
2009-09-28 23:16:17 -03:00 |
|
Pablo Hoffman
|
7428e4cebf
|
splitted make_release.sh in: build_release.sh and sign_release.sh
--HG--
rename : extras/make_release.sh => extras/build_release.sh
|
2009-09-28 23:07:00 -03:00 |
|
Pablo Hoffman
|
1dcb9ed72f
|
Merge with stable repo
|
2009-09-28 22:53:05 -03:00 |
|
Pablo Hoffman
|
504293b8c4
|
renamed tag: 0.7.0-rc1 to 0.7-rc1
|
2009-09-28 22:29:23 -03:00 |
|
Pablo Hoffman
|
2be739964d
|
bumped version to 0.7 final
|
2009-09-28 22:28:49 -03:00 |
|
Daniel Grana
|
e0ccb2cceb
|
ignore request if spider is not opened in downloader when called from engine.download
|
2009-09-28 12:44:05 -03:00 |
|
Daniel Grana
|
749b136595
|
Automated merge with ssh://hg.scrapy.org/scrapy-stable
|
2009-09-28 12:40:16 -03:00 |
|
Daniel Grana
|
1f9e0f10eb
|
don't log ignored requests errors at images pipeline
|
2009-09-28 12:38:49 -03:00 |
|
Daniel Grana
|
706cb439c5
|
Automated merge with ssh://hg.scrapy.org/scrapy-stable
|
2009-09-23 14:54:33 -03:00 |
|
Daniel Grana
|
c795771282
|
remove unused line from project settings template
|
2009-09-23 14:53:40 -03:00 |
|
Daniel Grana
|
bd0dc21e8d
|
Automated merge with ssh://hg.scrapy.org/scrapy-stable
|
2009-09-23 14:28:41 -03:00 |
|
Daniel Grana
|
dfd35c5280
|
Add common javascript mimetypes to be detected as TextResponses, also improve mimetype detection to avoid querying twice classes dict
|
2009-09-23 14:26:58 -03:00 |
|
Daniel Grana
|
bae726a172
|
Automated merge with ssh://hg.scrapy.org/scrapy-stable
|
2009-09-22 11:33:15 -03:00 |
|
Ismael Carnales
|
ef0e5d368f
|
updated spider templates to reflect the new spider callback return policy
|
2009-09-22 11:25:42 -03:00 |
|
Ismael Carnales
|
5862ba7db7
|
modified doc to reflect the new spider callback return policy (lists not needed)
|
2009-09-22 11:25:40 -03:00 |
|
Ismael Carnales
|
802f918b69
|
removed obsolete doc static file
|
2009-09-22 11:25:38 -03:00 |
|
Daniel Grana
|
337253f967
|
Automated merge with file:///home/dan/src/hg/scrapy-stable
|
2009-09-21 13:59:46 -03:00 |
|
Daniel Grana
|
6467c7055d
|
check iterable output of crawl spider and add tests for common function used to iterate spider output
|
2009-09-21 13:57:16 -03:00 |
|
Pablo Hoffman
|
201ce525e5
|
make_release.sh: disabled bdist_wininst command as it doesn't package data files properly on Linux - refs #109
|
2009-09-21 13:07:52 -03:00 |
|
Pablo Hoffman
|
12ebcfd6c4
|
bumped version to 0.8
|
2009-09-18 15:31:54 -03:00 |
|
Pablo Hoffman
|
b6155801f0
|
Added tag 0.7.0-rc1 for changeset f4a14daa04e6
|
2009-09-18 00:52:41 -03:00 |
|
Pablo Hoffman
|
bc778a0e5b
|
make_release.sh: added warning about broken bdist_wininst
0.7-rc1
|
2009-09-17 15:28:36 -03:00 |
|
Daniel Grana
|
1b28858bf9
|
improve XmlRpcRequest and update tests to cover more cases
|
2009-09-17 14:26:42 -03:00 |
|
Pablo Hoffman
|
48c3fb3ace
|
made XmlRpcRequestTest inherit from RequestTest and removed redundant test
|
2009-09-17 12:21:43 -03:00 |
|
Pablo Hoffman
|
6845505e7a
|
added make_release.sh script
|
2009-09-17 11:46:47 -03:00 |
|
Pablo Hoffman
|
6e10914454
|
added pypi classifiers to setup.py
|
2009-09-17 11:22:48 -03:00 |
|
Pablo Hoffman
|
6e93872955
|
updated installation guide for using releases
|
2009-09-17 11:06:55 -03:00 |
|
Pablo Hoffman
|
653db1d673
|
added bitmap for windows installer
|
2009-09-17 02:01:40 -03:00 |
|
Pablo Hoffman
|
132557dd14
|
some deployment changes in preparation for the 0.7.0 release candidate
|
2009-09-16 22:40:36 -03:00 |
|
Ismael Carnales
|
fd41f06056
|
added doc on how to enable an Item Pipeline component
|
2009-09-16 14:19:16 -03:00 |
|
Ismael Carnales
|
404e7e09d7
|
changed spider doc references in BaseSpider class
|
2009-09-16 14:10:11 -03:00 |
|
Daniel Grana
|
062730cbd8
|
fix csv exporter documentation
|
2009-09-16 00:17:50 -03:00 |
|
Daniel Grana
|
08aedcbe76
|
Automated merge with ssh://hg.scrapy.org/scrapy
|
2009-09-15 09:35:24 -03:00 |
|
Daniel Grana
|
e8514445dc
|
bugfix image thumb when no thumb is configured
|
2009-09-15 09:35:21 -03:00 |
|
Pablo Hoffman
|
02e228ad76
|
added support for returning deferreds in spider manager close_domain() method, and making sure engine_stopped signal is always sent (even when no spiders have run)
|
2009-09-15 09:27:30 -03:00 |
|
Pablo Hoffman
|
66ff3b3013
|
renamed defer_failed to defer_fail
|
2009-09-15 08:44:31 -03:00 |
|
Pablo Hoffman
|
56b292e057
|
XmlItemExporter: added built-in support for exporting multi-valued fields (for convenience)
|
2009-09-14 22:05:52 -03:00 |
|
Pablo Hoffman
|
e8960bf616
|
added runspider command to run spiders directly, without having to create a project
|
2009-09-14 22:05:14 -03:00 |
|
Pablo Hoffman
|
fcbbb5001e
|
ported spiderctl web console extensin to work with new core based on spider references
|
2009-09-14 20:35:47 -03:00 |
|
Pablo Hoffman
|
47aa716630
|
adapted web console to use unix timestamps for uptime instead of datetime
|
2009-09-14 20:35:07 -03:00 |
|
Pablo Hoffman
|
bc463bc9e8
|
using time.time() instead of datetime.utcnow() in engine.start_time atttribute
|
2009-09-14 20:28:46 -03:00 |
|
Pablo Hoffman
|
7e07f76edd
|
made pending_spiders attribute protected in spider scheduler
|
2009-09-14 20:27:54 -03:00 |
|
Pablo Hoffman
|
9b68432624
|
commented out line that was preventing errors from propagating to the request errback
|
2009-09-14 12:27:29 -03:00 |
|
Pablo Hoffman
|
2322312c63
|
Logging requests instead of responses in 'Crawled ...' messages
|
2009-09-14 10:31:29 -03:00 |
|
Pablo Hoffman
|
6f64dfe579
|
renamed spider manager close_domain() method to close_spider()
|
2009-09-14 10:06:54 -03:00 |
|
Pablo Hoffman
|
99467d4e6e
|
Changed (unstable) scheduler middleware API to receive spider (instead of domain) in enqueue_request method
|
2009-09-13 20:51:43 -03:00 |
|
Pablo Hoffman
|
00873cd16c
|
Another Spider Manager simplification: removed add_spider() method
|
2009-09-12 21:27:58 -03:00 |
|
Pablo Hoffman
|
dc82550058
|
Do not impose an arbitrary encoding in spider templates, because we don't know beforehand what enconding our users will use in their editors.
|
2009-09-12 21:04:21 -03:00 |
|