BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-25 18:24:30 +00:00
Code Releases Wiki Activity
1,188 Commits 39 Branches 119 Tags
Commit Graph

1188 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ismael Carnales
63150e2ed4 added item_to_dict util function 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40988
 2009-03-16 01:12:57 +00:00
Daniel Grana
f9e5915fd4 newitem: change itemadaptor tests to avoid false positives 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40987
 2009-03-13 20:05:37 +00:00
Andres Moreira
cbf6d76c5d Improved PriorityQueue and PriorityStack with a patch sent by Federico Feroldi. The new implementation is about two times faster than the old. 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40986
 2009-03-12 19:06:57 +00:00
Ismael Carnales
518a7c87dd make field_adaptors non-public 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40985
 2009-03-11 13:26:29 +00:00
Ismael Carnales
2ba017b14d add tests for adaptation of multivaluated fields 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40984
 2009-03-11 13:01:26 +00:00
Daniel Grana
0be771152b stats: fix stats key typo 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40983
 2009-03-11 03:01:30 +00:00
Pablo Hoffman
f8ce241484 fixed MailSender documentation example - closes #74 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40982
 2009-03-10 19:56:22 +00:00
Ismael Carnales
ec1a6af04a added MultiValuedField to newitem 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40981
 2009-03-10 19:35:45 +00:00
Ismael Carnales
2f75c2c93e removed unneeded import 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40980
 2009-03-10 19:17:53 +00:00
Ismael Carnales
04baa7aed7 removed declarative from newitem (now using 'plain' metaclasses) added basic test for newitem 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40979
 2009-03-10 18:40:26 +00:00
Ismael Carnales
774e7f623a fixed default adaptor inheritance, added test 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40978
 2009-03-10 17:59:53 +00:00
Daniel Grana
67ab30a5ba core: add application/xhtml+xml support to responsetypes 
more info at http://www.w3.org/TR/xhtml-media-types/#media-types

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40977
 2009-03-10 17:51:03 +00:00
Ismael Carnales
63e6ce6e47 made IDENTITY a function of adaptor module 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40976
 2009-03-10 16:57:27 +00:00
Ismael Carnales
698e6e73ea moved default_adaptor logic fully to the metaclass in ItemAdaptor 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40975
 2009-03-10 16:34:48 +00:00
Ismael Carnales
06e049e551 move meta class initialization to metaclass in ItemAdaptor 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40974
 2009-03-10 16:15:49 +00:00
Ismael Carnales
f105359666 use metaclass __getattr__ to return default_adaptor in ItemAdaptor 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40973
 2009-03-10 15:53:42 +00:00
Ismael Carnales
8bda81c01d add inheritance of item and adaptors to itemadaptor test 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40972
 2009-03-10 14:03:27 +00:00
Ismael Carnales
f9aa0a6a01 removed is_adaptor check in ItemAdaptor in favour of funcion adaptizer 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40971
 2009-03-10 13:38:32 +00:00
Pablo Hoffman
4093a4b83e added support to run scrapy tests with: python -m scrapy.tests 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40970
 2009-03-05 11:12:07 +00:00
Daniel Grana
e3c9f5951a contrib: fix a header bug in httpcompression middleware and add tests 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40969
 2009-03-04 20:11:12 +00:00
Ismael Carnales
8ad4211e26 small changes to make code 3.0 compatible :) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40968
 2009-03-04 17:11:20 +00:00
Ismael Carnales
ec22373b5a use ItemAdaptor.__classinit__ to set field_adaptors, also make defaulted field adaptors static_methods 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40967
 2009-03-03 22:35:15 +00:00
Ismael Carnales
b5b6b0cefc removed unneeded decorator from Item xD 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40966
 2009-03-03 21:32:37 +00:00
Ismael Carnales
154dded1de using declarative in ItemAdaptor, more to come 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40965
 2009-03-03 21:29:00 +00:00
Ismael Carnales
e744e87517 code style changes for newitem, and only return fields or special attrs in Item 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40964
 2009-03-03 21:18:40 +00:00
Damian Canabal
d26b968f51 fixing typo 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40963
 2009-03-03 21:14:50 +00:00
Ismael Carnales
60de639434 added declarative module for declarative style classses and make Item use it 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40962
 2009-03-03 21:12:37 +00:00
Daniel Grana
0d47fdf6e8 utils: revert canonicalize_url implementation, it wasnt intended to be changed 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40961
 2009-03-03 15:18:17 +00:00
Daniel Grana
8724ee1674 utils: url canonicalization keeps blanks query parameters 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40960
 2009-03-03 15:05:50 +00:00
Ismael Carnales
2e1a170563 added LxmlParserLinkExtractor a new try on link extracting 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40959
 2009-03-03 07:42:54 +00:00
Daniel Grana
309bf9277e xmlrpc: send request as POST by default and use dont_filter 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40958
 2009-03-03 05:37:41 +00:00
Daniel Grana
ff7c6d3782 http: add XmlRpcRequest based on xmlrpclib 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40957
 2009-03-03 00:37:06 +00:00
Daniel Grana
ab3fdbdb8e core: prevent download timeout from been accidentally disabled 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40956
 2009-03-02 23:22:36 +00:00
Ismael Carnales
875e8e57d3 added lxml based link extractor for supporting sites with broken htm 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40955
 2009-03-02 19:49:42 +00:00
Ismael Carnales
127e5ae029 convert process_attr to a parameter in contructor so extending the class is not needed 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40954
 2009-03-02 18:03:48 +00:00
Daniel Grana
1f38460ba4 allow returning None and non-iterables from spiders 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40953
 2009-03-02 17:29:48 +00:00
Daniel Grana
2acdd9b409 utils: add any value to iterable function 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40952
 2009-03-02 17:29:18 +00:00
Ismael Carnales
315934bf6b added new LinkExtractor based on HTMLParser and with a hook for preprocessing the attribute 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40951
 2009-03-02 17:27:29 +00:00
Daniel Grana
57c7c5e9f6 images: log a short message when images request is ignored to dupefilter 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40950
 2009-03-02 15:37:57 +00:00
Pablo Hoffman
4b0d69cc6d removed some unused imports 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40949
 2009-03-02 14:52:34 +00:00
Ismael Carnales
d8f31f5678 pass the item_instance of ItemAdaptor as an adaptor_arg 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40948
 2009-03-02 02:58:10 +00:00
Ismael Carnales
16c20f5c4c added default_adaptor to ItemAdaptor 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40947
 2009-03-01 14:26:11 +00:00
Ismael Carnales
f145e850ad discover adaptors in the python way 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40946
 2009-03-01 14:16:05 +00:00
Ismael Carnales
41ef6c262e added default_adaptor to the newitem documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40945
 2009-03-01 13:09:54 +00:00
Ismael Carnales
6724819423 updated newitem proposed docs 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40944
 2009-03-01 12:59:28 +00:00
Ismael Carnales
ff950e7aa9 added intro/tutorial for new Item and ItemAdaptor 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40943
 2009-03-01 12:28:55 +00:00
Daniel Grana
43b87acd81 docs: rename setting doc accordly 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40942
 2009-03-01 11:14:30 +00:00
Daniel Grana
e998f92297 docs: document SCHEDULER_MIDDLEWARE setting 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40941
 2009-03-01 11:04:09 +00:00
Pablo Hoffman
ee80b1bcc0 added doc about 'dont_merge_cookies' recently added by Damian 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40940
 2009-02-28 03:04:31 +00:00
Pablo Hoffman
ca22e332d1 renamed to_list function to arg_to_list, added docstring and one more test 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40939
 2009-02-28 02:51:18 +00:00