BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-28 10:44:39 +00:00
Code Releases Wiki Activity
814 Commits 39 Branches 119 Tags
Commit Graph

694 Commits

Author SHA1 Message Date
Pablo Hoffman
abdf4dee50 added doc about scrapy architecture. closes #31 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40664
 2009-01-06 23:05:04 +00:00
olveyra
4286e5e47c fix to revision 660 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40663
 2009-01-06 19:43:58 +00:00
Pablo Hoffman
1d36c3f0dd updated robotstxt, spidermw and downloadermw docs 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40662
 2009-01-06 17:34:14 +00:00
Pablo Hoffman
e9d050b91d renamed spider middleware methods to more consistent ones: process_spider_input, process_spider_output, process_spider_exception 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40661
 2009-01-06 17:30:29 +00:00
Pablo Hoffman
cf34fd4c98 added spider middleware documentation. closes #28 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40660
 2009-01-06 16:53:49 +00:00
Pablo Hoffman
7f7196c150 removed unused import 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40659
 2009-01-06 16:44:06 +00:00
Pablo Hoffman
5e63690939 added DEFAULT_SPIDER to default_settings and fixed formatting in DEFAULT_SPIDER setting documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40658
 2009-01-06 16:42:59 +00:00
elpolilla
c80fef7eb0 Added the example project from the tutorial 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40657
 2009-01-06 16:08:03 +00:00
Pablo Hoffman
9d6defa643 doc: fixed intro-install xref 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40656
 2009-01-06 14:34:26 +00:00
Pablo Hoffman
9da9041248 updated INSTALL file to point to docs/intro/install. also, utf-8 default encoding is no longer required 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40655
 2009-01-06 00:19:51 +00:00
Pablo Hoffman
3badafc8a7 updated installation doc 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40654
 2009-01-06 00:15:55 +00:00
Pablo Hoffman
c9d91106d8 updated documentation index 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40653
 2009-01-06 00:15:30 +00:00
Pablo Hoffman
4df3118c03 added note to scrapy version 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40652
 2009-01-05 23:49:06 +00:00
Pablo Hoffman
b3c7203ad9 updated some doc configuration - version, release and html_copy_source 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40651
 2009-01-05 23:29:25 +00:00
olveyra
9b857124c1 added DEFAULT_SPIDER setting help 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40650
 2009-01-05 18:11:48 +00:00
elpolilla
111ba5d583 Updated items documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40649
 2009-01-05 16:55:20 +00:00
elpolilla
f1962c1c0a . Moved init and repr methods from RobustScrapedItem to ScrapedItem 
. Refactored ScrapedItem's attribute method, added docstring and tests

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40648
 2009-01-05 16:54:54 +00:00
Pablo Hoffman
3c4012f924 updated scrapy version 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40647
 2009-01-05 03:37:06 +00:00
Pablo Hoffman
5c9c82d055 some improvements to selectors doc structure, added literalincludes for sample1.html (to avoid duplicating the content), renamed that file and moved to _static (so it appears on built doc), moved comments out of source code snippets and into documentation text, and splitted them. converted to required '>>>' console format, and added proper highlighting hints 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40646
 2009-01-05 02:49:23 +00:00
Pablo Hoffman
e61b581890 fixed wrong xref name 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40644
 2009-01-04 19:03:15 +00:00
samus_
2ea631398e reconfigured scrapy Response in order to disallow direct use of ResponseBody class that should be private 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40643
 2009-01-04 18:23:12 +00:00
elpolilla
58e4cf74b6 Added Items documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40642
 2009-01-04 15:37:43 +00:00
elpolilla
dc1a168d53 Fixed unquote adaptor's test 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40641
 2009-01-04 14:46:13 +00:00
elpolilla
604c86b779 Removed unappropiate unquote adaptor's default behaviour 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40640
 2009-01-04 14:25:15 +00:00
elpolilla
e4bfb27f83 Removed already-deprecated response decompression tool 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40639
 2009-01-04 10:50:14 +00:00
Pablo Hoffman
8d255f7bac updated some docs paths 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40638
 2009-01-04 04:54:06 +00:00
elpolilla
ea439e4b03 Updated Selectors documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40637
 2009-01-04 01:15:08 +00:00
elpolilla
a5025899ab Updated Spiders documentation 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40636
 2009-01-04 00:36:16 +00:00
Pablo Hoffman
f8f0db8bd3 doc: several more improvements 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40635
 2009-01-03 09:14:52 +00:00
Pablo Hoffman
137429e518 doc: added README 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40634
 2009-01-03 07:54:33 +00:00
Pablo Hoffman
16f9f5a9ef doc: added topic about robots.txt, added ROBOTSTXT_OBEY setting, added missing REQUESTS_PER_DOMAIN setting 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40633
 2009-01-03 07:41:43 +00:00
Pablo Hoffman
b59acb18ff minor typo and grammar corrections to docs/faq 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40632
 2009-01-03 07:40:40 +00:00
Pablo Hoffman
4a3ba6957b updated default_settings with robots.txt related settings 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40631
 2009-01-03 07:36:35 +00:00
Pablo Hoffman
e437ff67ae finished working version of robots.txt downloader middleware, and renamed the module/class 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40630
 2009-01-03 07:35:30 +00:00
Pablo Hoffman
8a404c4480 removed deprecated Request.__getitem__ method 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40629
 2009-01-03 06:15:30 +00:00
Pablo Hoffman
8d1d5102be updated downloader-middleware to link to Request and Response classes 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40628
 2009-01-03 03:11:08 +00:00
Pablo Hoffman
8b10d0183a added (incomplete) request-response doc 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40627
 2009-01-03 03:10:14 +00:00
Pablo Hoffman
44f6358ace renamed itempipeline.rst to item-pipeline.rst 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40626
 2009-01-03 03:05:19 +00:00
Pablo Hoffman
1a9845b3e8 added documentation for downloader middleware. closes #27 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40625
 2009-01-03 01:29:09 +00:00
Pablo Hoffman
093c7e121b docs: more updates to Makefile and conf.py 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40624
 2009-01-03 01:25:03 +00:00
Pablo Hoffman
14697b5fba added FAQ to docs 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40623
 2009-01-03 01:24:18 +00:00
Pablo Hoffman
d06fc99101 more stripping down of doc Makefile 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40622
 2009-01-03 00:58:09 +00:00
Pablo Hoffman
144eaa64b4 added stripped down version of Python documentation Makefile 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40621
 2009-01-03 00:43:04 +00:00
Pablo Hoffman
541451afb3 simplified obfuscated implementation of duplicated links removal and made it use scrapy.utils.python.unique function. also added missing 'unique' argument to docstring 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40620
 2009-01-02 20:45:34 +00:00
Pablo Hoffman
c69d94fb45 removed unused private variable 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40619
 2009-01-02 20:44:07 +00:00
Pablo Hoffman
f8a6e2fc7c added key argument to unique function 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40618
 2009-01-02 20:42:20 +00:00
Pablo Hoffman
371890008b removed ugly apostrophe 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40615
 2009-01-02 20:06:17 +00:00
Pablo Hoffman
fc36ad002f renamed ItemPipeline class to ItemPipelineManager to avoid confusions 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40614
 2009-01-02 19:59:41 +00:00
Pablo Hoffman
dd2f92f040 added comment about new docs 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40613
 2009-01-02 19:55:22 +00:00
Pablo Hoffman
e95a28ef3d added documentation for item pipeline, signals and exceptions. changed scrapy version to 0.7 in sphinx conf. closes #26, #40, #41 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40612
 2009-01-02 19:48:31 +00:00