BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-03-01 20:07:26 +00:00
Code Releases Wiki Activity
2,123 Commits 39 Branches 119 Tags
Commit Graph

2123 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Pablo Hoffman
f6eac8e348 Splitted single Debian package into two packages (closes #187): 
- scrapy: which provides only the library and scrapy-ctl command
- scrapy-service: which provides the service, upstart script, system user, etc

This allows a clean install of just the library for those which are not
interested in the Scrapy service.

--HG--
rename : debian/scrapy.dirs => debian/scrapy-service.dirs
rename : debian/scrapy.install => debian/scrapy-service.install
rename : debian/scrapy.postinst => debian/scrapy-service.postinst
rename : debian/scrapy.postrm => debian/scrapy-service.postrm
rename : debian/scrapy.upstart => debian/scrapy-service.upstart
rename : debian/conf/service_conf.py => debian/service_conf.py
 2010-07-31 15:50:12 -03:00
Ismael Carnales
e145ec686c Replaced default spider manager (TwistedPluginSpiderManger) with a simpler one that doesn't depend on Twisted Plugins infrastructure. 2010-07-30 17:30:32 -03:00
Pablo Hoffman
65c3de8e6a Rewritten walk_modules function to support eggs, and added tests 2010-07-30 17:05:55 -03:00
Ismael Carnales
6d06824488 utils: Add walk_packages utility function 
--HG--
rename : scrapy/tests/test_utils_misc.py => scrapy/tests/test_utils_misc/__init__.py
 2010-07-30 15:53:24 -03:00
Ismael Carnales
9511a3154e Fix error message when spider not found in parse command 2010-07-30 14:49:07 -03:00
Pablo Hoffman
e112def754 removed old untested module: scrapy.utils.mysql 2010-07-29 12:11:07 -03:00
Pablo Hoffman
6312826220 removed (no longer supported) webconsole code 2010-07-29 12:03:02 -03:00
molveyra
936ffe5e26 Automated merge with ssh://hg@hg.scrapy.org:2222/scrapy 2010-07-28 11:28:52 -03:00
molveyra
8781ef3914 Remove restriction of marking ignore-beneath only for img unpaired tags 2010-07-28 11:16:42 -03:00
Pablo Hoffman
2349d241e0 removed custom Makefile and version based on mercurial revision 2010-07-24 17:02:08 -03:00
Pablo Hoffman
e2290a5359 Some changes to Crawl spider: 
* added process_request attribute to rules
* removed docstrings, since it duplicates documentation
 2010-07-22 18:40:35 -03:00
Daniel Grana
4e2859e5d5 Automated merge with ssh://hg.scrapy.org/scrapy-0.9 2010-07-20 15:47:46 -03:00
Daniel Grana
68c7ef7d98 fix scraper leak closing spider. closes #182 2010-07-20 15:47:07 -03:00
Daniel Grana
3e013f564b update docs for defaultheaders middleware and change spider attribute to match global setting name 2010-07-16 16:17:08 -03:00
Daniel Grana
6883a99c1e Automated merge with ssh://hg.scrapy.org/scrapy-0.9 2010-07-16 14:56:00 -03:00
Daniel Grana
b799e5ee37 Support default headers per spider. closes #181 
--HG--
extra : rebase_source : 60162dffa4fbab525501e46b479dc272b8998942
 2010-07-16 14:51:14 -03:00
Pablo Hoffman
b91d40ba78 Fixed grammar error in doc (patch by stav) - closes #176 2010-07-16 11:34:18 -03:00
Pablo Hoffman
b8aa74ee9e bugfix in request_httprepr() function 2010-07-15 12:04:55 -03:00
Martin Olveyra
ec850b9fd1 Fix memusage report concatenation 2010-07-14 18:47:09 -03:00
Pablo Hoffman
90a04f0530 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-07-13 19:47:55 -03:00
Pablo Hoffman
cc32f6ec66 Applied patch to ClientForm to fix bug with wrong entities. Also added tests and left patch in repo in case we upgrade ClientForm in the future and need to re-apply it 2010-07-13 19:46:53 -03:00
Pablo Hoffman
9e37ec4230 fixed documentation typo (closes #151) 2010-07-13 19:03:02 -03:00
Ping Yin
b3a65d3313 HTTPCACHE: Don't cache response with codes in HTTPCACHE_IGNORE_HTTP_CODES 2010-07-09 13:14:25 -03:00
Pablo Hoffman
2067bcd8d0 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-07-08 14:03:58 -03:00
Juan Picca
2ddbbc8152 allow passing custom headers in FormRequest.from_response() 2010-07-08 14:02:28 -03:00
Pablo Hoffman
a6a86d9b4a Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-07-01 11:48:36 -03:00
Martin Olveyra
b258fc3305 Fixed bug with float values in meta refresh 2010-07-01 11:46:06 -03:00
Pablo Hoffman
3e976e5005 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-06-28 00:55:35 -03:00
Pablo Hoffman
b23af5ccf6 Added tag 0.9 for changeset 5caf3dc10a92 2010-06-28 00:54:47 -03:00
Pablo Hoffman
6525d3fe8c bumped version to 0.9 final 0.9 2010-06-28 00:54:43 -03:00
Pablo Hoffman
66053291d6 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-06-27 19:57:54 -03:00
Pablo Hoffman
861c04fb7e made encoding explicit in test_get_meta_refresh, to avoid depending on 
unreliable UnicodeDammit criteria.
 2010-06-27 19:55:39 -03:00
Pablo Hoffman
696bb63ac5 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-06-27 19:32:47 -03:00
Pablo Hoffman
22555df56a response_httprepr: fixed error with unknown response codes (closes #169) 2010-06-27 19:32:26 -03:00
Pablo Hoffman
5408202697 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-06-27 19:23:41 -03:00
Ismael Carnales
2571e1b7aa docs: Some DjangoItem docs improvements, closes #134. Thanks tn! 2010-06-27 09:09:54 -03:00
Daniel Grana
7b80cce148 Automated merge with ssh://hg.scrapy.org/scrapy 2010-06-25 13:06:14 -03:00
Daniel Grana
9148321765 do not redirect when there is a commented meta refresh header. closes #170 2010-06-25 12:57:15 -03:00
Pablo Hoffman
a61df93a22 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-06-22 14:00:51 -03:00
Pablo Hoffman
3bbad36998 Raise when trying to set an item field value using setattr api, and added tests. 2010-06-22 14:00:31 -03:00
Pablo Hoffman
504f90c6f7 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-06-22 13:39:29 -03:00
Pablo Hoffman
baa523055f removed nltk dependency from IBL code 2010-06-22 13:38:32 -03:00
Pablo Hoffman
e665e5abb7 bumped version to 0.10-dev 2010-06-14 22:00:54 -03:00
Pablo Hoffman
8815de94ff Added tag 0.9-rc1 for changeset 8b9c31e18c08 2010-06-14 18:37:55 -03:00
Pablo Hoffman
61e374eb56 bumped version to 0.9-rc1 0.9-rc1 2010-06-14 18:37:52 -03:00
Pablo Hoffman
115e9f2162 Added FAQ entry about running Scrapy deployment. 2010-06-14 18:21:12 -03:00
Daniel Grana
f3d2ee41cc mediapipeline: bugfix error raised when media requests has not callbacks, remove item_media_{downloaded,failed} hooks in favour or request.{errback,calback}, and add tests 
--HG--
extra : rebase_source : 72172406ab4ffc748e1648b46fe976e403b87c29
 2010-06-14 12:34:52 -03:00
Pablo Hoffman
0731c42e24 added "hg purge" to make tarball 2010-06-14 09:58:17 -03:00
Pablo Hoffman
4262a0af8c moved sign_release.sh code to Makefile 2010-06-14 09:49:07 -03:00
Pablo Hoffman
d8ac4857a5 a couple of fixes to make tests pass on win32 2010-06-14 09:11:35 -03:00