BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-26 12:43:48 +00:00
Code Releases Wiki Activity
989 Commits 39 Branches 119 Tags
Commit Graph

989 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Pablo Hoffman
d376b20996 minor fix to doctest 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40839
 2009-02-10 04:46:07 +00:00
Pablo Hoffman
90c1597cef fixed bug with items_to_csv and Excel. thanks Mat for the patch. See: 
http://groups.google.com/group/scrapy-developers/browse_thread/thread/9e2b80d226333011

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40838
 2009-02-10 02:43:19 +00:00
Daniel Grana
1b2de80124 remove scrapy/dos external referenced from scrapy.org site 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40837
 2009-02-09 17:20:41 +00:00
Ismael Carnales
844b59b5b3 reduced introduction text in proposed doc 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40836
 2009-02-09 15:06:22 +00:00
Ismael Carnales
d98f35af94 added items section (ScrapedItems) to proposed doc 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40835
 2009-02-09 14:29:48 +00:00
Ismael Carnales
3a3723ba2e removed reference to guid in ScrapedItem 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40834
 2009-02-09 13:59:15 +00:00
Pablo Hoffman
38c62ca357 settings must be documented in alphabetical order 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40833
 2009-02-08 21:00:26 +00:00
Ismael Carnales
45340d3d87 better index for proposed documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40832
 2009-02-06 20:19:17 +00:00
Ismael Carnales
5ca4728805 added complete spiders topic and reference (in one file) using an autodoc and manual doc mix 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40831
 2009-02-06 20:15:34 +00:00
Ismael Carnales
3fb945be7b fixed doc on BaseSpider 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40830
 2009-02-06 20:02:13 +00:00
Ismael Carnales
485b9c9ed5 updated docstrings of feedspiders for autodoc 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40829
 2009-02-06 20:01:27 +00:00
Ismael Carnales
9ba07653c6 added a proposed introduction (with all the main topics) to the proposed documentation, this would be the start for the new tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40828
 2009-02-06 16:20:49 +00:00
Ismael Carnales
be04a7f328 modify docs configuration for autodoc to work 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40827
 2009-02-05 15:39:52 +00:00
Ismael Carnales
96be669af2 fix tutorial1 errors 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40826
 2009-02-05 13:44:32 +00:00
Daniel Grana
fe2f018b1a FormRequest: urlencode multiples values of a single key using doseq 
this prevents urllib.urlencode from sending the repr of the value when
it founds a list or tuple.

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40825
 2009-02-05 13:41:10 +00:00
Ismael Carnales
6650e56354 changed representation of the project tree in the tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40824
 2009-02-05 13:35:14 +00:00
Daniel Grana
8f349737f8 add missing dashes to docs 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40823
 2009-02-05 12:43:30 +00:00
Daniel Grana
bc2ea867ca Adds docs for PROJECT_NAME setting. refs #58 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40822
 2009-02-05 12:39:43 +00:00
Daniel Grana
bf87fb8455 add proper docstring to string_camelcase. refs #58 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40821
 2009-02-05 12:20:25 +00:00
Ismael Carnales
751a844e36 updated tutorial to reflect project's structure change 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40820
 2009-02-05 12:14:49 +00:00
Daniel Grana
f7dac0e449 Creates an usable project structure on startproject. closes #58 and closes #54 
--HG--
rename : scrapy/trunk/scrapy/conf/project_template/__init__.py => scrapy/trunk/scrapy/templates/project/module/__init__.py
rename : scrapy/trunk/scrapy/conf/project_template/items.py => scrapy/trunk/scrapy/templates/project/module/items.py
rename : scrapy/trunk/scrapy/conf/project_template/scrapy_settings.py => scrapy/trunk/scrapy/templates/project/module/settings.py
rename : scrapy/trunk/scrapy/conf/project_template/spiders/__init__.py => scrapy/trunk/scrapy/templates/project/module/spiders/__init__.py
rename : scrapy/trunk/scrapy/conf/project_template/templates/spider_basic.tmpl => scrapy/trunk/scrapy/templates/project/module/templates/spider_basic.tmpl
rename : scrapy/trunk/scrapy/conf/project_template/templates/spider_crawl.tmpl => scrapy/trunk/scrapy/templates/project/module/templates/spider_crawl.tmpl
rename : scrapy/trunk/scrapy/conf/project_template/templates/spider_csvfeed.tmpl => scrapy/trunk/scrapy/templates/project/module/templates/spider_csvfeed.tmpl
rename : scrapy/trunk/scrapy/conf/project_template/templates/spider_xmlfeed.tmpl => scrapy/trunk/scrapy/templates/project/module/templates/spider_xmlfeed.tmpl
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40819
 2009-02-05 12:12:15 +00:00
Ismael Carnales
bccef8a463 fixed basespider links in topics/spider 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40818
 2009-01-31 06:09:50 +00:00
Ismael Carnales
fa8ebd1147 corrected naming and references in ref/spiders doc 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40817
 2009-01-31 06:01:33 +00:00
Pablo Hoffman
6220609718 added bin/runtests.sh script to run tests 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40816
 2009-01-30 23:20:49 +00:00
Pablo Hoffman
20a61caa2c fixed some doc typos 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40815
 2009-01-30 23:20:21 +00:00
Pablo Hoffman
005a642240 added SETTINGS_DISABLED environment variable to turn off custom settings (and only use Scrapy defaults) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40814
 2009-01-30 23:18:23 +00:00
Pablo Hoffman
16efbf87fc added some references to documentation and fixed some doc typos (thanks Patrick) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40813
 2009-01-30 22:33:50 +00:00
Pablo Hoffman
c672223091 splitted spiders doc from link extractor docs, moved the corresponding parts to ref and topics 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40812
 2009-01-30 21:53:40 +00:00
Pablo Hoffman
483ef3ba7f changed typo in Items docs 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40811
 2009-01-30 21:52:41 +00:00
Ismael Carnales
a834eef6c9 added BaseSpider attributes and method references and metadata (from tutorial) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40810
 2009-01-30 19:24:43 +00:00
Ismael Carnales
7e6c9c2e25 formatting changes and references to spiders added in the tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40809
 2009-01-30 19:14:16 +00:00
Ismael Carnales
2526e11be0 changed formatting a little bit and added class references to spiders topic 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40808
 2009-01-30 19:12:09 +00:00
Ismael Carnales
591a4a2bb5 added information on default item in new tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40807
 2009-01-30 14:45:52 +00:00
Ismael Carnales
f27d07c1a4 added warning of spider naming in new tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40806
 2009-01-30 14:26:40 +00:00
Ismael Carnales
3b012d3981 modified default project template structure in proposed tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40805
 2009-01-30 13:15:50 +00:00
elpolilla
648753190c Several corrections made to items and adaptors documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40804
 2009-01-30 13:05:52 +00:00
Ismael Carnales
d9a90a1f3f added proposed documentation, moved tutorial there 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40803
 2009-01-30 11:29:55 +00:00
elpolilla
6cc806a843 Added adaptors diagram svg to the repo 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40802
 2009-01-30 11:19:20 +00:00
Pablo Hoffman
bcf6e5476b added one more case to responsetypes 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40801
 2009-01-30 11:14:08 +00:00
elpolilla
769519c2aa Added missing adaptors diagram to documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40800
 2009-01-30 10:57:12 +00:00
Pablo Hoffman
fb13d3d83b doc: added status argument to domain_closed signal 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40799
 2009-01-29 20:18:53 +00:00
Daniel Grana
b3050ca0e4 pipeline: remove open_domain/close_domain hooks, use domain_open and domain_closed signals instead. docs updated. 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40798
 2009-01-29 19:18:03 +00:00
Pablo Hoffman
73c924643c bundled mime.types file with scrapy for platforms that have poor support for mime.types (windows), added from_args() convenience method to ResponseTypes 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40797
 2009-01-29 18:52:46 +00:00
Daniel Grana
f4e6ed3c2f run without RobotTxtMiddleware by default and update docs 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40796
 2009-01-29 18:22:59 +00:00
Ismael Carnales
e6375c6a5b added some info on items pipeline on the tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40795
 2009-01-29 17:57:21 +00:00
elpolilla
9634af9aa7 Updated adaptors and items documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40794
 2009-01-29 17:46:32 +00:00
Ismael Carnales
ff88171b05 added items to tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40793
 2009-01-29 14:35:54 +00:00
Ismael Carnales
04fc0471ec first version of the new tutorial (work in progress) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40792
 2009-01-29 12:29:28 +00:00
elpolilla
3ca2187e3b Removed non-generic implementation of _add_single_attributes and modified test 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40791
 2009-01-29 02:21:08 +00:00
elpolilla
bc4fcbeab7 Added response type recognition to local file urls 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40790
 2009-01-28 11:45:39 +00:00