BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-27 16:44:08 +00:00
Code Releases Wiki Activity
622 Commits 39 Branches 119 Tags
Commit Graph

622 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Pablo Hoffman
fbec2b3a43 moved scrapy.core.mail module to scrapy.mail 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40272
 2008-09-23 21:52:05 +00:00
Pablo Hoffman
cb8e0d9bdd added scrapy.utils.defer, moved deferred functions from scrapy.utils.misc to that module 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40271
 2008-09-23 21:48:48 +00:00
Pablo Hoffman
6637d8cf68 removed location_str function incorrectly added to this module 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40270
 2008-09-23 21:34:05 +00:00
samus_
dbfc6341c2 removed duplicated function convert_entity from scrapy.utils.misc 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40269
 2008-09-23 20:36:32 +00:00
german
4842fe0679 removed unquote_html 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40268
 2008-09-22 18:04:30 +00:00
Matias Aguirre
2b6ed80a66 Change site-media with static in css file 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40267
 2008-09-19 20:00:08 +00:00
Matias Aguirre
e985b6c368 Comment blog templatetags calls 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40266
 2008-09-19 19:57:45 +00:00
Matias Aguirre
d3f66b7d39 Remove blog url an installed app setting 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40265
 2008-09-19 19:53:57 +00:00
Matias Aguirre
bd28045b99 Remove blog application which is not django 1.0 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40264
 2008-09-19 19:19:28 +00:00
Matias Aguirre
4d388c946e Django 1.0 support over download application 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40263
 2008-09-19 19:17:41 +00:00
Daniel Grana
62975817af remove decobot reference and fix test_plugin 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40262
 2008-09-19 16:57:57 +00:00
Daniel Grana
baabdb004d remove decobot reference 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40261
 2008-09-19 16:57:21 +00:00
Daniel Grana
d87f979d2a grrr.. stupid bugfix 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40260
 2008-09-19 15:21:47 +00:00
eduardo
c05dcdab1d removed references to python2.5 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40259
 2008-09-19 14:11:14 +00:00
olveyra
ddd7393a0a reverted changeset 254 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40258
 2008-09-19 12:50:19 +00:00
olveyra
332460851b ouch! reverted a failed commit and now yes, removed an unneeded None 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40257
 2008-09-18 20:19:26 +00:00
olveyra
133e9242d3 removed unneeded None 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40256
 2008-09-18 20:13:31 +00:00
olveyra
6ce7ae4df8 allow to set download delay as a scrapy setting 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40255
 2008-09-18 20:11:30 +00:00
eduardo
cfb56e39bb bump version 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40254
 2008-09-18 19:21:40 +00:00
eduardo
c6d4a37a98 include templates 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40253
 2008-09-18 17:48:35 +00:00
Daniel Grana
818d9baf69 new setup.py 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40252
 2008-09-18 16:08:33 +00:00
eduardo
a85e53053d in the way of setuptooling 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40251
 2008-09-18 14:58:43 +00:00
Daniel Grana
93c313853d use request.deferred for cached valued too 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40250
 2008-09-17 12:34:48 +00:00
Daniel Grana
6f2d36199b media: use deferred of request to allow adding callback from get_media_requests 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40249
 2008-09-16 23:40:48 +00:00
Daniel Grana
7932a8a5b6 add image pipeline that stores and can thumbs images in different sizes 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40248
 2008-09-16 23:40:06 +00:00
Daniel Grana
4807f2eb82 move media_to_download hook to allow caching and prevent downloading 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40247
 2008-09-16 23:39:26 +00:00
Daniel Grana
5048491f8d mediapipeline: change cache variable by domaininfo 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40246
 2008-09-16 19:31:25 +00:00
Daniel Grana
41a4f22c27 messy typo adding errback 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40245
 2008-09-16 14:03:39 +00:00
anibal
e0d90f9d87 Scrapy architecture diagram 1st version, dia source 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40244
 2008-09-16 11:56:15 +00:00
Pablo Hoffman
0a7a5e5b3d added docs dir, guess for what? :) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40243
 2008-09-16 11:30:22 +00:00
Pablo Hoffman
b57148dfad renamed scrapy.lib to scrapy.xlib 
--HG--
rename : scrapy/trunk/scrapy/lib/BeautifulSoup.py => scrapy/trunk/scrapy/xlib/BeautifulSoup.py
rename : scrapy/trunk/scrapy/lib/__init__.py => scrapy/trunk/scrapy/xlib/__init__.py
rename : scrapy/trunk/scrapy/lib/lrucache.py => scrapy/trunk/scrapy/xlib/lrucache.py
rename : scrapy/trunk/scrapy/lib/lsprofcalltree.py => scrapy/trunk/scrapy/xlib/lsprofcalltree.py
rename : scrapy/trunk/scrapy/lib/pydispatch/__init__.py => scrapy/trunk/scrapy/xlib/pydispatch/__init__.py
rename : scrapy/trunk/scrapy/lib/pydispatch/dispatcher.py => scrapy/trunk/scrapy/xlib/pydispatch/dispatcher.py
rename : scrapy/trunk/scrapy/lib/pydispatch/errors.py => scrapy/trunk/scrapy/xlib/pydispatch/errors.py
rename : scrapy/trunk/scrapy/lib/pydispatch/license.txt => scrapy/trunk/scrapy/xlib/pydispatch/license.txt
rename : scrapy/trunk/scrapy/lib/pydispatch/robust.py => scrapy/trunk/scrapy/xlib/pydispatch/robust.py
rename : scrapy/trunk/scrapy/lib/pydispatch/robustapply.py => scrapy/trunk/scrapy/xlib/pydispatch/robustapply.py
rename : scrapy/trunk/scrapy/lib/pydispatch/saferef.py => scrapy/trunk/scrapy/xlib/pydispatch/saferef.py
rename : scrapy/trunk/scrapy/lib/spidermonkey/INSTALL.scrapy => scrapy/trunk/scrapy/xlib/spidermonkey/INSTALL.scrapy
rename : scrapy/trunk/scrapy/lib/spidermonkey/__init__.py => scrapy/trunk/scrapy/xlib/spidermonkey/__init__.py
rename : scrapy/trunk/scrapy/lib/spidermonkey/sm_settings.py => scrapy/trunk/scrapy/xlib/spidermonkey/sm_settings.py
rename : scrapy/trunk/scrapy/lib/spidermonkey/spidermonkey.py => scrapy/trunk/scrapy/xlib/spidermonkey/spidermonkey.py
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40242
 2008-09-15 15:37:44 +00:00
Pablo Hoffman
2cb300b926 removed simplejson from scrapy code 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40241
 2008-09-15 15:29:12 +00:00
Daniel Grana
08feb4df1c ername methods and enforce extraction of Request objects as item media to download 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40240
 2008-09-15 13:55:23 +00:00
Pablo Hoffman
0090fcb0f5 improved fix to support IE7 (thanks bill\!) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40239
 2008-09-15 13:08:44 +00:00
Pablo Hoffman
d7a88e8d77 fixed hack for IE6 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40238
 2008-09-15 12:58:19 +00:00
Pablo Hoffman
31d3b2cf5a added hack for IE6 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40237
 2008-09-15 12:56:31 +00:00
Daniel Grana
4944762341 move bugtraps to avoid silence of bugs in media_downloaded or media_failed 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40236
 2008-09-15 12:46:34 +00:00
Daniel Grana
ca695c8c79 small bugfix and add bugtraps 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40235
 2008-09-15 08:57:51 +00:00
Daniel Grana
6407a744e3 remove item.image_urls from get_urls_from_item 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40234
 2008-09-15 08:27:15 +00:00
Daniel Grana
af0808be95 add info object to public methods 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40233
 2008-09-15 07:08:27 +00:00
Daniel Grana
4263c3a2e1 Adds docstrings to public methods, remove unused imports, normalize urls to requests. 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40232
 2008-09-15 06:01:40 +00:00
Daniel Grana
31e497e5eb add media_to_download hook support, and return cached result if available 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40231
 2008-09-15 04:33:40 +00:00
olveyra
df2f4b21b8 removed reference to guid (decobot specific) attribute in item 
pipeline logs

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40230
 2008-09-15 01:23:10 +00:00
Daniel Grana
3803f67645 mediamiddleware: bugfixes and remove prints 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40229
 2008-09-12 21:55:14 +00:00
olveyra
2fd5cf644e - Support for not output in cluster status 
- schedule option by default does not have output

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40228
 2008-09-12 20:48:48 +00:00
Daniel Grana
d612701b66 add media pipeline skeleton 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40227
 2008-09-12 19:58:45 +00:00
Andres Moreira
2dc20bb56e Fixed condition of new option --quiet. 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40226
 2008-09-11 16:35:50 +00:00
olveyra
4cffc039ff - leave in adaptors.py only the elemental adaptors pipeline code. 
- moved to contrib/adaptorpipeline the advanced adaptors pipeline code.

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40225
 2008-09-11 15:33:35 +00:00
Andres Moreira
23814537b3 Fixed bug in the total items count. Added new option --quiet to the action diff. This option doesn't show the report if there aren't differences. 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40224
 2008-09-11 14:59:16 +00:00
elpolilla
2b6bcd61df Small piece of code moved for not complying with conventions 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40223
 2008-09-11 14:50:15 +00:00