BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-26 17:24:38 +00:00
Code Releases Wiki Activity
934 Commits 39 Branches 119 Tags
Commit Graph

934 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ismael Carnales
3a3723ba2e removed reference to guid in ScrapedItem 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40834
 2009-02-09 13:59:15 +00:00
Pablo Hoffman
38c62ca357 settings must be documented in alphabetical order 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40833
 2009-02-08 21:00:26 +00:00
Ismael Carnales
45340d3d87 better index for proposed documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40832
 2009-02-06 20:19:17 +00:00
Ismael Carnales
5ca4728805 added complete spiders topic and reference (in one file) using an autodoc and manual doc mix 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40831
 2009-02-06 20:15:34 +00:00
Ismael Carnales
3fb945be7b fixed doc on BaseSpider 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40830
 2009-02-06 20:02:13 +00:00
Ismael Carnales
485b9c9ed5 updated docstrings of feedspiders for autodoc 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40829
 2009-02-06 20:01:27 +00:00
Ismael Carnales
9ba07653c6 added a proposed introduction (with all the main topics) to the proposed documentation, this would be the start for the new tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40828
 2009-02-06 16:20:49 +00:00
Ismael Carnales
be04a7f328 modify docs configuration for autodoc to work 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40827
 2009-02-05 15:39:52 +00:00
Ismael Carnales
96be669af2 fix tutorial1 errors 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40826
 2009-02-05 13:44:32 +00:00
Daniel Grana
fe2f018b1a FormRequest: urlencode multiples values of a single key using doseq 
this prevents urllib.urlencode from sending the repr of the value when
it founds a list or tuple.

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40825
 2009-02-05 13:41:10 +00:00
Ismael Carnales
6650e56354 changed representation of the project tree in the tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40824
 2009-02-05 13:35:14 +00:00
Daniel Grana
8f349737f8 add missing dashes to docs 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40823
 2009-02-05 12:43:30 +00:00
Daniel Grana
bc2ea867ca Adds docs for PROJECT_NAME setting. refs #58 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40822
 2009-02-05 12:39:43 +00:00
Daniel Grana
bf87fb8455 add proper docstring to string_camelcase. refs #58 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40821
 2009-02-05 12:20:25 +00:00
Ismael Carnales
751a844e36 updated tutorial to reflect project's structure change 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40820
 2009-02-05 12:14:49 +00:00
Daniel Grana
f7dac0e449 Creates an usable project structure on startproject. closes #58 and closes #54 
--HG--
rename : scrapy/trunk/scrapy/conf/project_template/__init__.py => scrapy/trunk/scrapy/templates/project/module/__init__.py
rename : scrapy/trunk/scrapy/conf/project_template/items.py => scrapy/trunk/scrapy/templates/project/module/items.py
rename : scrapy/trunk/scrapy/conf/project_template/scrapy_settings.py => scrapy/trunk/scrapy/templates/project/module/settings.py
rename : scrapy/trunk/scrapy/conf/project_template/spiders/__init__.py => scrapy/trunk/scrapy/templates/project/module/spiders/__init__.py
rename : scrapy/trunk/scrapy/conf/project_template/templates/spider_basic.tmpl => scrapy/trunk/scrapy/templates/project/module/templates/spider_basic.tmpl
rename : scrapy/trunk/scrapy/conf/project_template/templates/spider_crawl.tmpl => scrapy/trunk/scrapy/templates/project/module/templates/spider_crawl.tmpl
rename : scrapy/trunk/scrapy/conf/project_template/templates/spider_csvfeed.tmpl => scrapy/trunk/scrapy/templates/project/module/templates/spider_csvfeed.tmpl
rename : scrapy/trunk/scrapy/conf/project_template/templates/spider_xmlfeed.tmpl => scrapy/trunk/scrapy/templates/project/module/templates/spider_xmlfeed.tmpl
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40819
 2009-02-05 12:12:15 +00:00
Ismael Carnales
bccef8a463 fixed basespider links in topics/spider 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40818
 2009-01-31 06:09:50 +00:00
Ismael Carnales
fa8ebd1147 corrected naming and references in ref/spiders doc 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40817
 2009-01-31 06:01:33 +00:00
Pablo Hoffman
6220609718 added bin/runtests.sh script to run tests 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40816
 2009-01-30 23:20:49 +00:00
Pablo Hoffman
20a61caa2c fixed some doc typos 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40815
 2009-01-30 23:20:21 +00:00
Pablo Hoffman
005a642240 added SETTINGS_DISABLED environment variable to turn off custom settings (and only use Scrapy defaults) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40814
 2009-01-30 23:18:23 +00:00
Pablo Hoffman
16efbf87fc added some references to documentation and fixed some doc typos (thanks Patrick) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40813
 2009-01-30 22:33:50 +00:00
Pablo Hoffman
c672223091 splitted spiders doc from link extractor docs, moved the corresponding parts to ref and topics 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40812
 2009-01-30 21:53:40 +00:00
Pablo Hoffman
483ef3ba7f changed typo in Items docs 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40811
 2009-01-30 21:52:41 +00:00
Ismael Carnales
a834eef6c9 added BaseSpider attributes and method references and metadata (from tutorial) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40810
 2009-01-30 19:24:43 +00:00
Ismael Carnales
7e6c9c2e25 formatting changes and references to spiders added in the tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40809
 2009-01-30 19:14:16 +00:00
Ismael Carnales
2526e11be0 changed formatting a little bit and added class references to spiders topic 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40808
 2009-01-30 19:12:09 +00:00
Ismael Carnales
591a4a2bb5 added information on default item in new tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40807
 2009-01-30 14:45:52 +00:00
Ismael Carnales
f27d07c1a4 added warning of spider naming in new tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40806
 2009-01-30 14:26:40 +00:00
Ismael Carnales
3b012d3981 modified default project template structure in proposed tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40805
 2009-01-30 13:15:50 +00:00
elpolilla
648753190c Several corrections made to items and adaptors documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40804
 2009-01-30 13:05:52 +00:00
Ismael Carnales
d9a90a1f3f added proposed documentation, moved tutorial there 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40803
 2009-01-30 11:29:55 +00:00
elpolilla
6cc806a843 Added adaptors diagram svg to the repo 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40802
 2009-01-30 11:19:20 +00:00
Pablo Hoffman
bcf6e5476b added one more case to responsetypes 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40801
 2009-01-30 11:14:08 +00:00
elpolilla
769519c2aa Added missing adaptors diagram to documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40800
 2009-01-30 10:57:12 +00:00
Pablo Hoffman
fb13d3d83b doc: added status argument to domain_closed signal 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40799
 2009-01-29 20:18:53 +00:00
Daniel Grana
b3050ca0e4 pipeline: remove open_domain/close_domain hooks, use domain_open and domain_closed signals instead. docs updated. 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40798
 2009-01-29 19:18:03 +00:00
Pablo Hoffman
73c924643c bundled mime.types file with scrapy for platforms that have poor support for mime.types (windows), added from_args() convenience method to ResponseTypes 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40797
 2009-01-29 18:52:46 +00:00
Daniel Grana
f4e6ed3c2f run without RobotTxtMiddleware by default and update docs 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40796
 2009-01-29 18:22:59 +00:00
Ismael Carnales
e6375c6a5b added some info on items pipeline on the tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40795
 2009-01-29 17:57:21 +00:00
elpolilla
9634af9aa7 Updated adaptors and items documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40794
 2009-01-29 17:46:32 +00:00
Ismael Carnales
ff88171b05 added items to tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40793
 2009-01-29 14:35:54 +00:00
Ismael Carnales
04fc0471ec first version of the new tutorial (work in progress) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40792
 2009-01-29 12:29:28 +00:00
elpolilla
3ca2187e3b Removed non-generic implementation of _add_single_attributes and modified test 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40791
 2009-01-29 02:21:08 +00:00
elpolilla
bc4fcbeab7 Added response type recognition to local file urls 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40790
 2009-01-28 11:45:39 +00:00
Andres Moreira
e9fbce4ee2 Improved code of utils.markup functions. 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40789
 2009-01-27 17:58:51 +00:00
elpolilla
c0601cd03e Refactored remove_entities and fixed the hex entities bug (they werent recognised). Also, tests were added 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40788
 2009-01-27 14:05:22 +00:00
Pablo Hoffman
c3d61dc999 changing extension in test from csv to txt (not all systems support the text/csv mimetype) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40787
 2009-01-27 13:33:28 +00:00
Pablo Hoffman
134b867abe added support for unknown file extensions to ResponseTypes.from_filename 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40786
 2009-01-27 13:08:07 +00:00
elpolilla
14e67d59c2 Added test for encoding in csviter 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40785
 2009-01-27 12:35:09 +00:00