BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-03-03 22:57:36 +00:00
Code Releases Wiki Activity
4,472 Commits 39 Branches 119 Tags
Commit Graph

4472 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Daniel Grana
bc2ea867ca Adds docs for PROJECT_NAME setting. refs #58 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40822
 2009-02-05 12:39:43 +00:00
Daniel Grana
bf87fb8455 add proper docstring to string_camelcase. refs #58 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40821
 2009-02-05 12:20:25 +00:00
Ismael Carnales
751a844e36 updated tutorial to reflect project's structure change 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40820
 2009-02-05 12:14:49 +00:00
Daniel Grana
f7dac0e449 Creates an usable project structure on startproject. closes #58 and closes #54 
--HG--
rename : scrapy/trunk/scrapy/conf/project_template/__init__.py => scrapy/trunk/scrapy/templates/project/module/__init__.py
rename : scrapy/trunk/scrapy/conf/project_template/items.py => scrapy/trunk/scrapy/templates/project/module/items.py
rename : scrapy/trunk/scrapy/conf/project_template/scrapy_settings.py => scrapy/trunk/scrapy/templates/project/module/settings.py
rename : scrapy/trunk/scrapy/conf/project_template/spiders/__init__.py => scrapy/trunk/scrapy/templates/project/module/spiders/__init__.py
rename : scrapy/trunk/scrapy/conf/project_template/templates/spider_basic.tmpl => scrapy/trunk/scrapy/templates/project/module/templates/spider_basic.tmpl
rename : scrapy/trunk/scrapy/conf/project_template/templates/spider_crawl.tmpl => scrapy/trunk/scrapy/templates/project/module/templates/spider_crawl.tmpl
rename : scrapy/trunk/scrapy/conf/project_template/templates/spider_csvfeed.tmpl => scrapy/trunk/scrapy/templates/project/module/templates/spider_csvfeed.tmpl
rename : scrapy/trunk/scrapy/conf/project_template/templates/spider_xmlfeed.tmpl => scrapy/trunk/scrapy/templates/project/module/templates/spider_xmlfeed.tmpl
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40819
 2009-02-05 12:12:15 +00:00
Ismael Carnales
bccef8a463 fixed basespider links in topics/spider 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40818
 2009-01-31 06:09:50 +00:00
Ismael Carnales
fa8ebd1147 corrected naming and references in ref/spiders doc 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40817
 2009-01-31 06:01:33 +00:00
Pablo Hoffman
6220609718 added bin/runtests.sh script to run tests 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40816
 2009-01-30 23:20:49 +00:00
Pablo Hoffman
20a61caa2c fixed some doc typos 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40815
 2009-01-30 23:20:21 +00:00
Pablo Hoffman
005a642240 added SETTINGS_DISABLED environment variable to turn off custom settings (and only use Scrapy defaults) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40814
 2009-01-30 23:18:23 +00:00
Pablo Hoffman
16efbf87fc added some references to documentation and fixed some doc typos (thanks Patrick) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40813
 2009-01-30 22:33:50 +00:00
Pablo Hoffman
c672223091 splitted spiders doc from link extractor docs, moved the corresponding parts to ref and topics 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40812
 2009-01-30 21:53:40 +00:00
Pablo Hoffman
483ef3ba7f changed typo in Items docs 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40811
 2009-01-30 21:52:41 +00:00
Ismael Carnales
a834eef6c9 added BaseSpider attributes and method references and metadata (from tutorial) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40810
 2009-01-30 19:24:43 +00:00
Ismael Carnales
7e6c9c2e25 formatting changes and references to spiders added in the tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40809
 2009-01-30 19:14:16 +00:00
Ismael Carnales
2526e11be0 changed formatting a little bit and added class references to spiders topic 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40808
 2009-01-30 19:12:09 +00:00
Ismael Carnales
591a4a2bb5 added information on default item in new tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40807
 2009-01-30 14:45:52 +00:00
Ismael Carnales
f27d07c1a4 added warning of spider naming in new tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40806
 2009-01-30 14:26:40 +00:00
Ismael Carnales
3b012d3981 modified default project template structure in proposed tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40805
 2009-01-30 13:15:50 +00:00
elpolilla
648753190c Several corrections made to items and adaptors documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40804
 2009-01-30 13:05:52 +00:00
Ismael Carnales
d9a90a1f3f added proposed documentation, moved tutorial there 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40803
 2009-01-30 11:29:55 +00:00
elpolilla
6cc806a843 Added adaptors diagram svg to the repo 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40802
 2009-01-30 11:19:20 +00:00
Pablo Hoffman
bcf6e5476b added one more case to responsetypes 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40801
 2009-01-30 11:14:08 +00:00
elpolilla
769519c2aa Added missing adaptors diagram to documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40800
 2009-01-30 10:57:12 +00:00
Pablo Hoffman
fb13d3d83b doc: added status argument to domain_closed signal 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40799
 2009-01-29 20:18:53 +00:00
Daniel Grana
b3050ca0e4 pipeline: remove open_domain/close_domain hooks, use domain_open and domain_closed signals instead. docs updated. 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40798
 2009-01-29 19:18:03 +00:00
Pablo Hoffman
73c924643c bundled mime.types file with scrapy for platforms that have poor support for mime.types (windows), added from_args() convenience method to ResponseTypes 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40797
 2009-01-29 18:52:46 +00:00
Daniel Grana
f4e6ed3c2f run without RobotTxtMiddleware by default and update docs 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40796
 2009-01-29 18:22:59 +00:00
Ismael Carnales
e6375c6a5b added some info on items pipeline on the tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40795
 2009-01-29 17:57:21 +00:00
elpolilla
9634af9aa7 Updated adaptors and items documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40794
 2009-01-29 17:46:32 +00:00
Ismael Carnales
ff88171b05 added items to tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40793
 2009-01-29 14:35:54 +00:00
Ismael Carnales
04fc0471ec first version of the new tutorial (work in progress) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40792
 2009-01-29 12:29:28 +00:00
elpolilla
3ca2187e3b Removed non-generic implementation of _add_single_attributes and modified test 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40791
 2009-01-29 02:21:08 +00:00
elpolilla
bc4fcbeab7 Added response type recognition to local file urls 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40790
 2009-01-28 11:45:39 +00:00
Andres Moreira
e9fbce4ee2 Improved code of utils.markup functions. 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40789
 2009-01-27 17:58:51 +00:00
elpolilla
c0601cd03e Refactored remove_entities and fixed the hex entities bug (they werent recognised). Also, tests were added 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40788
 2009-01-27 14:05:22 +00:00
Pablo Hoffman
c3d61dc999 changing extension in test from csv to txt (not all systems support the text/csv mimetype) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40787
 2009-01-27 13:33:28 +00:00
Pablo Hoffman
134b867abe added support for unknown file extensions to ResponseTypes.from_filename 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40786
 2009-01-27 13:08:07 +00:00
elpolilla
14e67d59c2 Added test for encoding in csviter 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40785
 2009-01-27 12:35:09 +00:00
Pablo Hoffman
1e55e88111 fixed some typos in previous commit 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40784
 2009-01-27 12:18:23 +00:00
Pablo Hoffman
11a9236125 added example about creating Requests with cookies 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40783
 2009-01-27 12:17:40 +00:00
Pablo Hoffman
14ceca98bf added FormRequest example 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40782
 2009-01-27 12:10:49 +00:00
Pablo Hoffman
e25bfa5d88 added more cases to ResponseTypes, and tests for ResponseTypes 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40781
 2009-01-27 11:27:09 +00:00
elpolilla
b6246cbc37 Fixed encoding-related bug in csviter 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40780
 2009-01-27 11:10:42 +00:00
Pablo Hoffman
4013497edb more patches sent by Patrick 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40779
 2009-01-26 23:42:51 +00:00
Pablo Hoffman
c1a1b8945a some doc fixes suggested by Patrick Mézard 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40778
 2009-01-26 23:38:21 +00:00
Pablo Hoffman
57189e1b92 applied Patrick Mézard patch for loading local files 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40777
 2009-01-26 23:31:04 +00:00
Pablo Hoffman
ce1700dd8e added libxml2 installation steps for MacOSX (thanks Patrick Mézard) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40776
 2009-01-26 23:28:19 +00:00
Pablo Hoffman
279b9ac40d updated adaptors docs to reflect its inestability and improved the ReST formatting (80 column lines, no ugly pipes at the beginnig of lines) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40775
 2009-01-26 23:22:53 +00:00
elpolilla
e8e87bcb52 Fixed small bug in adaptor pipelines that tried to work with None values 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40774
 2009-01-26 16:20:58 +00:00
Pablo Hoffman
7ed88fd0f3 added Content-Disposition encoding discovery to ResponseTypes 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40773
 2009-01-26 15:36:53 +00:00