BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-28 15:34:23 +00:00
Code Releases Wiki Activity
2,335 Commits 39 Branches 119 Tags
Commit Graph

2335 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Pablo Hoffman
0ffc8d29b6 Don't silence warnings after log is started. Closes #246 2010-09-22 01:00:34 -03:00
Pablo Hoffman
4c61df7abb get_spider_list_from_eggfile(): fixed bug when SCRAPY_SETTINGS_DISABLED is set 2010-09-20 08:47:55 -03:00
Pablo Hoffman
8329491296 Added Shuaib Khan to AUTHORS 2010-09-20 08:34:22 -03:00
Shuaib
9288f622f9 Added formname parameter for FormRequest.from_response 2010-09-20 08:33:24 -03:00
Pablo Hoffman
400c4134af Make scrapyd.eggutils compatible with Python 2.5 and added tests for get_spider_list_from_eggfile() function (closes #242) 2010-09-19 21:08:27 -03:00
Ping Yin
aa86f180b9 BaseSgmlLinkExtractor: Fix extract '>>' as '>' 
The anchor text is extracted as '>' in
<a href="/">&gt;&gt;</a>

Signed-off-by: Ping Yin <pkufranky@gmail.com>
 2010-09-19 19:28:35 -03:00
Ping Yin
f41fd3214e test_engine.py: fix typo brwoser => browser 
Signed-off-by: Ping Yin <pkufranky@gmail.com>
 2010-09-19 19:28:13 -03:00
Pablo Hoffman
08952562e9 Make Ubuntu package conflict with standard Scrapy Debian/Ubuntu package 2010-09-17 16:25:02 -03:00
Daniel Grana
cad077574b Fix black shapes appearing round transparent images 2010-09-16 14:19:32 -03:00
Pablo Hoffman
4cecbcdc5b Fixed bug in Scrapyd launcher when running projects without eggs. Refs #238 2010-09-15 21:03:43 -03:00
Pablo Hoffman
a5908f052f Bumped version to 0.10.3 2010-09-15 21:03:21 -03:00
Pablo Hoffman
b33c8e49e2 Added tag 0.10.2 for changeset 68d08e757571 2010-09-15 14:48:47 -03:00
Pablo Hoffman
c01b8651ff Bumped version to 0.10.2 0.10.2 2010-09-15 14:04:33 -03:00
Pablo Hoffman
5fe9a49e51 Force recalculating the encoding on HTTPCompression middleware until we are sure the responsetypes guessing is reliable. Refs #239, #240 2010-09-15 13:57:17 -03:00
Pablo Hoffman
28f6bdf7bb Added tag 0.10.1 for changeset bb80107f959d 2010-09-15 02:44:41 -03:00
Pablo Hoffman
beee7e4fbc Merge with trunk 0.10.1 2010-09-15 01:50:42 -03:00
Pablo Hoffman
39499a2437 Fixed bug in HTTP Compression middleware which was failing to properly discover the encoding when the encoding was declared inside the response body. Closes #239. Also changed responsetypes to return Response class (instead of HtmlResponse) when the response has a Content-Encoding header 2010-09-14 20:22:25 -03:00
Pablo Hoffman
9acc99e723 Added 'inthread' decorator 2010-09-14 14:36:35 -03:00
Pablo Hoffman
c559b06a85 Removed unused import 2010-09-14 01:53:05 -03:00
Pablo Hoffman
833baa6041 Support running projects without eggs in Scrapyd. Closes #238 2010-09-14 01:44:25 -03:00
Pablo Hoffman
b76cd42690 Added tests for Scrapyd components. Closes #237 2010-09-14 01:44:10 -03:00
Pablo Hoffman
d0c7a946bd Bumped version to 0.10.1 2010-09-13 19:02:21 -03:00
Pablo Hoffman
1fa96dbcd3 Bumped version to 0.11 2010-09-13 19:02:15 -03:00
Pablo Hoffman
e2a3ab5141 Added tag 0.10 for changeset 3b32720c96a3 2010-09-13 18:49:33 -03:00
Pablo Hoffman
a3fc60cfcd Added missing mime.types files to MANIFEST.in 0.10 2010-09-11 04:30:08 -03:00
Pablo Hoffman
bf467fc37a Check 'dont_merge_cookies' membership in request.meta, instead of getting its value 2010-09-10 15:29:15 -03:00
Pablo Hoffman
e7a958a035 Fixed exception thrown when in FreeBSD when /proc exists but it's not mounted. Closes #235 2010-09-10 15:17:36 -03:00
Pablo Hoffman
b16eff227b Bumped version to 0.10 2010-09-10 01:48:45 -03:00
Pablo Hoffman
9b9b3a2b64 Removed unused import 2010-09-09 23:35:51 -03:00
Pablo Hoffman
7d14a52234 Reference dont_merge_cookies in list of special Request.meta keys 2010-09-09 21:54:26 -03:00
Pablo Hoffman
7f21a6384f Documented handle_httpstatus_list request.meta key 2010-09-09 21:50:40 -03:00
Pablo Hoffman
f1c943543a Added dont_retry request.meta key to make RetryMiddleware ignore requests. Closes #234 2010-09-09 21:43:44 -03:00
Pablo Hoffman
9f01e3e79e Added dont_redirect request.meta key to make RedirectMiddleware ignore requests. Closes #233 2010-09-09 21:37:35 -03:00
Pablo Hoffman
ad2b979e0f Changed TextResponse.replace() behaviour by keeping previous encoding when not specified 2010-09-09 18:01:00 -03:00
Pablo Hoffman
b9ea9dcf1a Restored wrongly removed import 2010-09-09 17:42:54 -03:00
Pablo Hoffman
c041328b45 Fixed bug with thread unsafety calls from Scrapy shell 2010-09-09 15:26:46 -03:00
Pablo Hoffman
24c0088bde Added tag 0.10-rc1 for changeset 131f8e906f75 2010-09-09 00:41:18 -03:00
Pablo Hoffman
38bbec8dca Bumped version to 0.10-rc1 0.10-rc1 2010-09-09 00:40:58 -03:00
Pablo Hoffman
fe77794f76 Added SpiderContext extension. Closes #203 2010-09-08 15:52:35 -03:00
Pablo Hoffman
1d5a7df125 Fixed encoding error with some item dropped messages. Closes #232 2010-09-08 15:34:13 -03:00
Pablo Hoffman
cb7cc7c8bb Fixed bugs with SQLiteDict and added tests 2010-09-08 14:32:28 -03:00
Pablo Hoffman
7da79b90fe Make url/body attributes of Request/Response objects read-only - use replace() to change them. Deprecation warning left for backwards compatibilty. 2010-09-08 00:15:11 -03:00
Pablo Hoffman
c1aab2f58e Copy callback/errback attributes when copying Requests 2010-09-08 00:15:09 -03:00
Pablo Hoffman
e9ebebb230 Removed UrlFilterMiddleware from scrapy.contrib - see this snippet for an alternative: http://snippets.scrapy.org/snippets/12/ 2010-09-07 17:51:02 -03:00
Pablo Hoffman
8ba6fa2096 Fixed bugs introduced by previous scheduler middleware change 2010-09-07 17:47:47 -03:00
Pablo Hoffman
dbc0ba7236 Added class constructors scrapy.contrib.ibl FieldDescriptor and ItemDescriptor 2010-09-07 14:26:50 -03:00
Pablo Hoffman
23b13f0cc8 Simplified SchedulerMiddlewareManager (to be removed in Scrapy 0.11) by making it inherit from MiddlewareManager 2010-09-07 13:10:49 -03:00
Pablo Hoffman
5918d0163e Automated merge with ssh://hg.scrapy.org:2222/scrapy 2010-09-07 13:03:16 -03:00
Pablo Hoffman
c40e5a7491 Moved module: scrapy.core.downloader.manager to scrapy.core.downloader 
--HG--
rename : scrapy/core/downloader/manager.py => scrapy/core/downloader/__init__.py
 2010-09-07 13:02:15 -03:00
Daniel Grana
12b04b068f make download_timeout configurable by request. closes #229 
--HG--
extra : rebase_source : e57dfd4aeb98d48b04fc4d0c6469e9a85e4b33a8
 2010-09-07 13:01:40 -03:00