BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-26 23:23:48 +00:00
Code Releases Wiki Activity
557 Commits 39 Branches 119 Tags
Commit Graph

557 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Pablo Hoffman
ed5e25c913 removed tagline from scrapy logo 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40557
 2008-12-27 18:21:37 +00:00
Pablo Hoffman
f932a3d842 added spider_exceptions to scrapy stats 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40556
 2008-12-27 17:37:05 +00:00
Pablo Hoffman
6fa33f4463 fixed minor bug in scrapy manager 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40555
 2008-12-27 02:06:05 +00:00
Pablo Hoffman
e89ad0be8c fixed minor bug in scrapy manager 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40554
 2008-12-27 02:05:05 +00:00
Pablo Hoffman
9557e24203 added start_requests method to BaseSpider, made start_urls empty by default instead of a required attribute in every spider 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40553
 2008-12-27 01:07:29 +00:00
elpolilla
5b159d0f04 - Improved unquote_markup by using generators instead of lists 
- Added possibility of specifying headers in items_to_csv

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40552
 2008-12-26 18:53:15 +00:00
elpolilla
f59f1c8bc0 Added items_to_csv function 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40551
 2008-12-26 16:10:03 +00:00
elpolilla
c7332cd372 Updated scrapy tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40550
 2008-12-26 14:03:45 +00:00
elpolilla
867ee20c66 Updated scrapy overview 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40549
 2008-12-26 12:21:53 +00:00
elpolilla
4ee3626763 Modified XMLFeedSpider in order to support parsing with HtmlXPathSelector 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40548
 2008-12-26 11:51:09 +00:00
elpolilla
b9360f659e Added tests for str_to_unicode and unicode_to_str 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40547
 2008-12-24 14:14:03 +00:00
elpolilla
86aa3bcb65 Added str_to_unicode and unicode_to_str functions, and used them in utils/markup 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40546
 2008-12-24 13:02:13 +00:00
elpolilla
95ed5d86a2 - Modified markup functions again; this time, in order to support utf-8 encoded strings (without interfering with unicode objects) 
- Reverted changes in tests

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40545
 2008-12-24 00:00:14 +00:00
elpolilla
e2b0870957 Fixed markup tests. Those functions must always work with unicode objects. 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40544
 2008-12-23 23:27:53 +00:00
elpolilla
f4cecafc02 Removed some bogus decoding of strings in several markup-handling functions. Input must always be passed in a unicode object 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40543
 2008-12-23 23:12:56 +00:00
Daniel Grana
6d33e12c8b aws: sign requests if url hostname ends with s3.amazonaws.com common bucket suffix. This allows a spider to send requests to S3 with out need to sign it inside spider code 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40542
 2008-12-23 14:31:58 +00:00
elpolilla
0cd074c94c Some changes made to shell command: 
. Removed traces of the old decompressor tool
 . Made the responses go through the downloader middlewares after being downloaded
 . Added the command its own settings file

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40541
 2008-12-23 14:21:46 +00:00
elpolilla
bd8bd25cd8 Removed unuseful and noisy test 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40540
 2008-12-23 01:37:32 +00:00
elpolilla
193fbfdc3d Updated scrapy's tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40539
 2008-12-19 14:46:56 +00:00
elpolilla
70af2e8e1d Modified Unquote adaptor to make use of the new unquote_markup function instead of remove_entities 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40538
 2008-12-19 11:09:51 +00:00
elpolilla
ac7ecb7810 Added adapt_response method to XMLFeedSpiders 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40537
 2008-12-19 10:16:58 +00:00
elpolilla
9d76ff606b Fixed bug in xmliter, which didnt add headers to the nodes it returned 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40536
 2008-12-19 10:16:31 +00:00
elpolilla
a98ba31816 Bugfix in parse command. Some options were not being processed 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40535
 2008-12-18 16:49:09 +00:00
Ismael Carnales
cf8b5fae37 massive rework of templates and styles 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40534
 2008-12-18 16:22:09 +00:00
elpolilla
bda22a539c Added more options to unquote_markup that are passed to remove_entities 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40533
 2008-12-18 16:12:49 +00:00
elpolilla
08150f5acb Added "unquote_markup" function to scrapy.utils.markup 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40532
 2008-12-18 16:03:46 +00:00
Ismael Carnales
5b1aeac3ec fixed typo in style 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40531
 2008-12-18 13:34:40 +00:00
Ismael Carnales
2fab6549d4 make menu float in the right of logo 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40530
 2008-12-18 13:33:21 +00:00
Ismael Carnales
4fd712e1e5 added new logo, also make it clickable 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40529
 2008-12-18 13:06:13 +00:00
Ezequiel Rivero
4856e2d64c fix of the footer. 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40528
 2008-12-18 10:36:21 +00:00
elpolilla
1247193980 Updated scrapy tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40527
 2008-12-17 17:43:20 +00:00
elpolilla
770094697a Improved items set_adaptors method, and added a __repr__ method to AdaptorPipes 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40526
 2008-12-17 16:40:29 +00:00
Ismael Carnales
4a16bbf74c more margin makes it more legible 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40525
 2008-12-16 19:01:50 +00:00
Ismael Carnales
e7adff7ae2 removed footer margin 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40524
 2008-12-16 19:00:24 +00:00
Ismael Carnales
8671c33e4c now using left-aligned liquid layout 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40523
 2008-12-16 18:59:18 +00:00
Daniel Grana
570f49d407 stats: limit the default number of stats history to 10 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40522
 2008-12-16 18:49:12 +00:00
Ismael Carnales
7414e93102 added breadcrumb navigation to docs 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40521
 2008-12-16 16:57:52 +00:00
Ismael Carnales
ed93eec374 removed border from last menu item 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40520
 2008-12-16 16:55:29 +00:00
Ismael Carnales
f3d6d71150 make home and download validate xhtml strict, watch out please 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40519
 2008-12-16 16:52:08 +00:00
Ismael Carnales
a8b2f0c7e9 added charset 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40518
 2008-12-16 16:46:28 +00:00
Ismael Carnales
e3d1afe98c fixed html, is good to close the lis xD 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40517
 2008-12-16 16:42:02 +00:00
Ismael Carnales
58ce43c14e moved parents on top of prev/next links 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40516
 2008-12-16 16:18:34 +00:00
Ismael Carnales
89de849ed1 updated install link 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40515
 2008-12-16 16:17:29 +00:00
Ismael Carnales
6766a39628 added view to serve images 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40514
 2008-12-16 16:14:15 +00:00
Ismael Carnales
aab5a4a7dd added links and reformated some texts 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40513
 2008-12-16 15:53:01 +00:00
Ismael Carnales
e140b1e435 completed the install doc 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40512
 2008-12-16 15:40:27 +00:00
Ismael Carnales
a026e382ed renamed intro to overview 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40511
 2008-12-16 15:02:32 +00:00
Ismael Carnales
e1ea75baac archived old documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40510
 2008-12-16 15:01:26 +00:00
Ismael Carnales
99f956c166 making dir for old docs 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40509
 2008-12-16 15:00:35 +00:00
Ismael Carnales
de003d15cd renamed tutorial2 to tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40508
 2008-12-16 14:59:38 +00:00