Mikael Åhlén
|
22da1783bd
|
added a test-case for wrong quotechar
|
2014-09-13 03:47:40 +02:00 |
|
Mikael Åhlén
|
47b6dff9f1
|
Allow to specify the quotechar in CSVFeedSpider
|
2014-09-13 02:14:57 +02:00 |
|
Daniel Graña
|
5bcabfe9c9
|
SPIDER_MODULES can be set as a csv string
|
2014-09-10 23:25:57 -03:00 |
|
Daniel Graña
|
c05e99a4f4
|
oops, restore Pillow from precise test requirements
|
2014-09-10 12:21:08 -03:00 |
|
Daniel Graña
|
a823207f18
|
Stop logobserver only when set
|
2014-09-10 12:09:07 -03:00 |
|
Daniel Graña
|
ec93c0fdcc
|
Add the tests changes for previous commit
|
2014-09-10 12:05:18 -03:00 |
|
Daniel Graña
|
ce180227fa
|
Twisted 11.1.0 (precise) can not deal with generators in DeferredList
Also create a list of the crawlers before iterating them because crawlers are removed from the set once stopped
|
2014-09-10 12:04:14 -03:00 |
|
Daniel Graña
|
99971dc8a8
|
Do not pop the crawler from the managed list
|
2014-09-09 20:59:07 +00:00 |
|
Daniel Graña
|
774aa9ee56
|
Merge branch 'crawler-api'
|
2014-09-09 17:23:11 -03:00 |
|
Daniel Graña
|
8ddf0811a8
|
Correctly detect when all managed crawlers are done in CrawlerRunner
|
2014-09-09 17:21:39 -03:00 |
|
Daniel Graña
|
68954fa503
|
Merge pull request #879 from Curita/fix-874-issue
Fix #874 issue
|
2014-09-08 14:32:06 -03:00 |
|
Julia Medina
|
51532af69a
|
Erase unneeded flag in CrawlerProcess.start
|
2014-09-07 13:03:34 -03:00 |
|
Julia Medina
|
d513b5a542
|
Run root logger in CrawlerProcess creation instead of in its start method
|
2014-09-07 13:02:39 -03:00 |
|
Pablo Hoffman
|
acb0a61cf1
|
Merge pull request #871 from eltermann/master
Removed unused 'load=False' parameter from walk_modules()
|
2014-09-02 12:55:04 -03:00 |
|
eltermann
|
1dff1fbf75
|
Removed unused 'load=False' parameter from walk_modules()
|
2014-09-02 08:33:36 -03:00 |
|
Daniel Graña
|
5daa14770b
|
Merge branch 'Curita-per-spider-settings'
|
2014-09-01 21:58:13 -03:00 |
|
Julia Medina
|
c2592b39fd
|
Test verifying that CrawlerRunner populates spider class settings
|
2014-09-01 21:56:57 -03:00 |
|
Julia Medina
|
77bd26a66d
|
Non mutable default in Spider.custom_settings
|
2014-09-01 21:56:57 -03:00 |
|
Julia Medina
|
16e62e9c9b
|
Per-spider settings documentation
|
2014-09-01 21:56:57 -03:00 |
|
Julia Medina
|
9ef3972cfb
|
Per-spider settings tests
|
2014-09-01 21:56:57 -03:00 |
|
Julia Medina
|
4932ec43a7
|
Per-spider settings implementation
|
2014-09-01 21:56:57 -03:00 |
|
Daniel Graña
|
ccde3317d7
|
Merge pull request #816 from Curita/api-cleanup
GSoC API cleanup
|
2014-09-01 21:55:36 -03:00 |
|
Pablo Hoffman
|
620dbe7116
|
Merge pull request #865 from yakxxx/master
SgmlLinkExtractor - fix for parsing <area> tag with Unicode present
|
2014-08-29 16:18:44 -03:00 |
|
Mikhail Korobov
|
8aa731c87f
|
Merge pull request #866 from adamdonahue/patch-1
Fix typo
|
2014-08-29 06:59:47 +06:00 |
|
Adam Donahue
|
d92914d297
|
Fix typo
|
2014-08-28 20:30:50 -04:00 |
|
yakxxx
|
e4689556f0
|
SgmlLinkExtractor - fix for parsing <area> tag with Unicode present
|
2014-08-28 18:55:58 +02:00 |
|
Mikhail Korobov
|
774ab74ad2
|
Merge pull request #864 from younghz/master
Duplicate comma in request-response.rst
|
2014-08-28 18:52:51 +06:00 |
|
Uyounghz
|
d49766a6ac
|
Duplicate comma in request-response.rst
|
2014-08-28 19:58:58 +08:00 |
|
Mikhail Korobov
|
c4f9e9d8e7
|
Merge pull request #856 from eliasdorneles/fix-embed-ipython-shell
fix IPython shell scope issue by using IPython.embed()
|
2014-08-20 00:16:33 +06:00 |
|
Pablo Hoffman
|
8360380db0
|
removed scrapy-ws.py, moved to scrapy-jsonrpc package
|
2014-08-19 10:30:25 -03:00 |
|
Elias Dorneles
|
6f50cf55a4
|
fix IPython shell scope issue and load IPython user config
|
2014-08-18 18:41:05 -03:00 |
|
Daniel Graña
|
841dd5f1f5
|
Update webservice.rst
|
2014-08-18 17:48:01 -03:00 |
|
Daniel Graña
|
d684ecad7b
|
Merge pull request #846 from rocioar/master
fix dont_merge_cookies bad behaviour when set to false on meta
|
2014-08-18 13:54:11 -03:00 |
|
Daniel Graña
|
94d00b2a26
|
Merge branch 'jsonrpc-split'
|
2014-08-15 23:28:26 -03:00 |
|
Daniel Graña
|
a9292cfab7
|
jsonrpc webservice moved to https://github.com/scrapy/scrapy-jsonrpc repository
|
2014-08-15 23:28:13 -03:00 |
|
Rocio Aramberri
|
51b0bd281d
|
fix dont settings on meta behaviour, add docs and tests
|
2014-08-15 13:47:42 -07:00 |
|
Julia Medina
|
6339864f95
|
Minor refactor in the docs and functions used in the shell command
|
2014-08-14 12:32:37 -03:00 |
|
Julia Medina
|
70f2010db1
|
Change error type when updating frozen settings
|
2014-08-14 11:59:25 -03:00 |
|
Julia Medina
|
3547ca6e61
|
Add example on running spiders outside projects
|
2014-08-14 11:50:33 -03:00 |
|
Julia Medina
|
419026615f
|
Deprecate Crawler.spiders attribute
|
2014-08-14 09:19:41 -03:00 |
|
Julia Medina
|
c90977ca98
|
Drop support for scrapy.project.crawler (And scrapy.stats consequently)
|
2014-08-12 14:02:56 -03:00 |
|
Julia Medina
|
9cbbfd8b04
|
Adjust spiders' utils to new SpiderManager API
|
2014-08-12 14:02:56 -03:00 |
|
Julia Medina
|
900a487682
|
Support multiple simultaneous LogObservers listening different crawlers
|
2014-08-12 14:02:56 -03:00 |
|
Julia Medina
|
89df18bd2b
|
Fix usage of crawler_process in ScrapyCommands
|
2014-08-12 14:02:56 -03:00 |
|
Julia Medina
|
d0edad4b0b
|
Drop support for ScrapyCommand.crawler property
|
2014-08-12 14:02:56 -03:00 |
|
Julia Medina
|
870438e5f4
|
Update tests utils, fixing get_crawler and removing docrawl
|
2014-08-12 14:02:56 -03:00 |
|
Julia Medina
|
d40273561d
|
CrawlerProcess cleanup changes
|
2014-08-12 14:02:55 -03:00 |
|
Julia Medina
|
980e30a187
|
Crawler interface cleanup
|
2014-08-12 14:02:55 -03:00 |
|
Julia Medina
|
d7038b2a13
|
SpiderManager interface cleanup
|
2014-08-12 14:02:55 -03:00 |
|
Julia Medina
|
39c6a80f9d
|
Both getdict and getlist return copies of the requested values
|
2014-08-12 14:02:55 -03:00 |
|