BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-26 20:03:44 +00:00
Code Releases Wiki Activity
5,143 Commits 39 Branches 119 Tags
Commit Graph

5143 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
olveyra
e447044fb9 reverted an experimental code that should have been commited 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40293
 2008-10-02 22:41:51 +00:00
olveyra
1f4e484a5c added DOWNLOAD_DELAY comment in settings template 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40292
 2008-10-02 20:39:36 +00:00
olveyra
58a419f45c added support for global DOWNLOAD_DELAY setting 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40291
 2008-10-02 19:59:51 +00:00
olveyra
3ec47301e3 allow to directly specify which domain corresponds to a given request 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40290
 2008-10-02 19:22:34 +00:00
Pablo Hoffman
98f3314bed simplified test without loosing functionality 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40289
 2008-09-30 19:57:05 +00:00
olveyra
9ba4573b13 added test for r284 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40288
 2008-09-30 19:24:24 +00:00
olveyra
c755f5a535 better management of some redirection loops. 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40287
 2008-09-30 16:56:12 +00:00
olveyra
433f35b417 small fix 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40286
 2008-09-30 16:19:20 +00:00
olveyra
a38c72bb05 safe_url_string should not escape unreserved marks (see RFC 2396, sec 
2.3)

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40285
 2008-09-30 14:15:23 +00:00
olveyra
3f32bae45f - copied original request to response.request in get_url method 
- deleted unused comment

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40284
 2008-09-30 13:44:11 +00:00
elpolilla
3ae4d62f38 - Added the posibility of knowing the decompressed response's format, in the decompression tool 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40283
 2008-09-29 18:14:28 +00:00
Damian Canabal
f8f2f3a542 rolled back public ent_re to private and added a function has_entities instead 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40282
 2008-09-29 13:21:35 +00:00
Damian Canabal
ad00d5e632 changed private html entity regex to public 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40281
 2008-09-29 12:52:54 +00:00
samus_
c07937c4df added comment 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40280
 2008-09-25 12:49:19 +00:00
samus_
c4aa1e7e8b small fix to the regex 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40279
 2008-09-25 12:45:43 +00:00
Daniel Grana
555cb0940b images: use brief exception logging 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40278
 2008-09-25 12:22:31 +00:00
olveyra
c73cbdd9b1 allow to override item class adaptor in constructor 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40277
 2008-09-25 02:14:02 +00:00
samus_
79485ceb5c added support for xml-declared encodings 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40276
 2008-09-24 18:19:19 +00:00
Damian Canabal
a4b709e03f added test for remove entities 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40275
 2008-09-24 12:40:55 +00:00
Damian Canabal
4e7e539b03 html entities regexp improvement 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40274
 2008-09-24 12:25:10 +00:00
Pablo Hoffman
79be2ca97d moved scrapy.core.log module to scrapy.log 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40273
 2008-09-23 22:18:20 +00:00
Pablo Hoffman
fbec2b3a43 moved scrapy.core.mail module to scrapy.mail 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40272
 2008-09-23 21:52:05 +00:00
Pablo Hoffman
cb8e0d9bdd added scrapy.utils.defer, moved deferred functions from scrapy.utils.misc to that module 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40271
 2008-09-23 21:48:48 +00:00
Pablo Hoffman
6637d8cf68 removed location_str function incorrectly added to this module 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40270
 2008-09-23 21:34:05 +00:00
samus_
dbfc6341c2 removed duplicated function convert_entity from scrapy.utils.misc 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40269
 2008-09-23 20:36:32 +00:00
german
4842fe0679 removed unquote_html 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40268
 2008-09-22 18:04:30 +00:00
Matias Aguirre
2b6ed80a66 Change site-media with static in css file 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40267
 2008-09-19 20:00:08 +00:00
Matias Aguirre
e985b6c368 Comment blog templatetags calls 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40266
 2008-09-19 19:57:45 +00:00
Matias Aguirre
d3f66b7d39 Remove blog url an installed app setting 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40265
 2008-09-19 19:53:57 +00:00
Matias Aguirre
bd28045b99 Remove blog application which is not django 1.0 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40264
 2008-09-19 19:19:28 +00:00
Matias Aguirre
4d388c946e Django 1.0 support over download application 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40263
 2008-09-19 19:17:41 +00:00
Daniel Grana
62975817af remove decobot reference and fix test_plugin 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40262
 2008-09-19 16:57:57 +00:00
Daniel Grana
baabdb004d remove decobot reference 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40261
 2008-09-19 16:57:21 +00:00
Daniel Grana
d87f979d2a grrr.. stupid bugfix 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40260
 2008-09-19 15:21:47 +00:00
eduardo
c05dcdab1d removed references to python2.5 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40259
 2008-09-19 14:11:14 +00:00
olveyra
ddd7393a0a reverted changeset 254 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40258
 2008-09-19 12:50:19 +00:00
olveyra
332460851b ouch! reverted a failed commit and now yes, removed an unneeded None 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40257
 2008-09-18 20:19:26 +00:00
olveyra
133e9242d3 removed unneeded None 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40256
 2008-09-18 20:13:31 +00:00
olveyra
6ce7ae4df8 allow to set download delay as a scrapy setting 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40255
 2008-09-18 20:11:30 +00:00
eduardo
cfb56e39bb bump version 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40254
 2008-09-18 19:21:40 +00:00
eduardo
c6d4a37a98 include templates 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40253
 2008-09-18 17:48:35 +00:00
Daniel Grana
818d9baf69 new setup.py 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40252
 2008-09-18 16:08:33 +00:00
eduardo
a85e53053d in the way of setuptooling 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40251
 2008-09-18 14:58:43 +00:00
Daniel Grana
93c313853d use request.deferred for cached valued too 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40250
 2008-09-17 12:34:48 +00:00
Daniel Grana
6f2d36199b media: use deferred of request to allow adding callback from get_media_requests 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40249
 2008-09-16 23:40:48 +00:00
Daniel Grana
7932a8a5b6 add image pipeline that stores and can thumbs images in different sizes 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40248
 2008-09-16 23:40:06 +00:00
Daniel Grana
4807f2eb82 move media_to_download hook to allow caching and prevent downloading 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40247
 2008-09-16 23:39:26 +00:00
Daniel Grana
5048491f8d mediapipeline: change cache variable by domaininfo 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40246
 2008-09-16 19:31:25 +00:00
Daniel Grana
41a4f22c27 messy typo adding errback 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40245
 2008-09-16 14:03:39 +00:00
anibal
e0d90f9d87 Scrapy architecture diagram 1st version, dia source 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40244
 2008-09-16 11:56:15 +00:00