BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-03-01 04:27:58 +00:00
Code Releases Wiki Activity
5,155 Commits 39 Branches 119 Tags
Commit Graph

1165 Commits

Author SHA1 Message Date
Pablo Hoffman
5835224eee Merge pull request #896 from scrapy/robotstxt-once 
[MRG] process robots.txt once
 2014-10-02 00:58:55 -03:00
Mikhail Korobov
6fcf9dce50 DOC document from_crawler method for item pipelines; add an example. 2014-09-25 03:13:51 +06:00
Mikhail Korobov
36eec8f413 dont_obey_robotstxt meta key; don't process requests to /robots.txt 2014-09-23 00:10:43 +06:00
Mikhail Korobov
bdbca1e2d7 DOC request queue memory usage 2014-09-21 07:30:44 +06:00
Mikhail Korobov
bc0f481a73 DOC bring back notes about multiple spiders per process because it is now documented how to do that 2014-09-21 07:12:01 +06:00
Mikhail Korobov
a122fdbfea Update leaks.rst: there is now only a single spider in a process. 2014-09-21 06:54:00 +06:00
Mikhail Korobov
e435b3e3a3 DOC simplify extension docs 2014-09-21 00:19:24 +06:00
John-Scott Atlakson
a312ebfb43 Update request-response.rst 
Fixed minor typo
 2014-09-14 22:06:31 +06:00
Mikael Åhlén
47b6dff9f1 Allow to specify the quotechar in CSVFeedSpider 2014-09-13 02:14:57 +02:00
Julia Medina
16e62e9c9b Per-spider settings documentation 2014-09-01 21:56:57 -03:00
Daniel Graña
ccde3317d7 Merge pull request #816 from Curita/api-cleanup 
GSoC API cleanup
 2014-09-01 21:55:36 -03:00
Mikhail Korobov
774ab74ad2 Merge pull request #864 from younghz/master 
Duplicate comma in request-response.rst
 2014-08-28 18:52:51 +06:00
Uyounghz
d49766a6ac Duplicate comma in request-response.rst 2014-08-28 19:58:58 +08:00
Daniel Graña
841dd5f1f5 Update webservice.rst 2014-08-18 17:48:01 -03:00
Daniel Graña
d684ecad7b Merge pull request #846 from rocioar/master 
fix dont_merge_cookies bad behaviour when set to false on meta
 2014-08-18 13:54:11 -03:00
Daniel Graña
a9292cfab7 jsonrpc webservice moved to https://github.com/scrapy/scrapy-jsonrpc repository 2014-08-15 23:28:13 -03:00
Rocio Aramberri
51b0bd281d fix dont settings on meta behaviour, add docs and tests 2014-08-15 13:47:42 -07:00
Julia Medina
3547ca6e61 Add example on running spiders outside projects 2014-08-14 11:50:33 -03:00
Julia Medina
419026615f Deprecate Crawler.spiders attribute 2014-08-14 09:19:41 -03:00
Julia Medina
c90977ca98 Drop support for scrapy.project.crawler (And scrapy.stats consequently) 2014-08-12 14:02:56 -03:00
Julia Medina
900a487682 Support multiple simultaneous LogObservers listening different crawlers 2014-08-12 14:02:56 -03:00
Julia Medina
d40273561d CrawlerProcess cleanup changes 2014-08-12 14:02:55 -03:00
Julia Medina
980e30a187 Crawler interface cleanup 2014-08-12 14:02:55 -03:00
Julia Medina
d7038b2a13 SpiderManager interface cleanup 2014-08-12 14:02:55 -03:00
Julia Medina
39c6a80f9d Both getdict and getlist return copies of the requested values 2014-08-12 14:02:55 -03:00
Julia Medina
3ae971468f Add Settings.copy, freeze and frozencopy method 2014-08-12 14:02:55 -03:00
Julia Medina
84fa004793 Add from_crawler class method to base Spider 2014-08-11 11:23:57 -03:00
Daniel Graña
8fece4b0b8 Add 0.24.4 release notes 2014-08-09 17:21:20 -03:00
Daniel Graña
4badcc077d Add 0.24.3 release notes 2014-08-09 00:31:26 -03:00
Daniel Graña
0772201ac8 Update installation docs 2014-08-08 18:07:38 -03:00
Daniel Graña
1a96cd5a3d update tox usage on contributing.rst 2014-07-31 12:28:35 -03:00
Daniel Graña
242c08553f move tests under root dir 2014-07-30 19:45:08 -03:00
Lisimia
b6b44a33f7 Update shell.rst typo 
There was a typo. 
"start" was mistyped as "star"
 2014-07-28 12:26:13 -04:00
Elias Dorneles
c298eaf164 removes weird indentation in the shell results 2014-07-23 20:42:36 -03:00
Elias Dorneles
514e8582d0 improved explanations, clarified blog post as source, added link for XPath string functions in the spec 2014-07-23 19:43:57 -03:00
Elias Dorneles
80de43ddab adding some xpath tips to selectors docs 2014-07-22 23:20:53 -03:00
Rolando Espinoza
e441a01818 Updated input/ouput processor example according to #560. 2014-07-16 09:04:48 -04:00
Roy Crihfield
1a950e8f5a Fix typo in commands.rst 2014-07-10 14:48:15 -03:00
Daniel Graña
379f6ee6b2 Add 0.24.2 release notes 2014-07-08 05:53:35 -03:00
Zbigniew Siciarz
0466e8cb7a Fixed Python syntax in tutorial. 2014-07-04 10:38:01 +02:00
Mikhail Korobov
00cd4f0fa5 deprecate SgmlLinkExtractor 2014-07-02 23:15:17 +06:00
Mikhail Korobov
173a9dfe86 DOC fix formatting in release notes 2014-06-28 13:28:57 +06:00
Daniel Graña
284a2a06c9 Add 0.24.1 release notes 2014-06-27 12:03:37 -03:00
Daniel Graña
77c8284fa5 set 0.24.0 release date 2014-06-26 11:45:32 -03:00
Daniel Graña
ee33efac39 Add 0.24.0 release notes 2014-06-26 10:29:47 -03:00
Daniel Graña
436c1c8980 address latest comments 2014-06-25 15:12:58 -03:00
Daniel Graña
2ad8db6ae6 Merge pull request #761 from dangra/lxmlextractor 
Promote LxmlLinkExtractor as LxmlExtractor
 2014-06-25 15:07:02 -03:00
Daniel Graña
a9ecef5662 promote LxmlLinkExtractor as default in docs 2014-06-25 14:34:30 -03:00
Daniel Graña
5b2faf61c3 recognize jl extension as jsonlines exporter and update docs 2014-06-25 13:55:15 -03:00
Daniel Graña
1b32ece918 Merge branch 'settings-cleanup' of https://github.com/Curita/scrapy into Curita-settings-cleanup 2014-06-25 02:55:20 -03:00