Mikhail Korobov
fee5565709
don't expose deprecated crawler.spiders attribute in telnet console
2015-04-17 01:25:48 +05:00
Mikhail Korobov
403e7c7c70
rename scrapy.spidermanager.SpiderManager to scrapy.spiderloader.SpiderLoader
2015-04-16 20:07:53 +05:00
Pablo Hoffman
fb85bd4b10
Merge pull request #1132 from sibiryakov/dns-options2
...
[MRG+1] Dns cache size and timeout options
2015-04-15 14:08:19 -03:00
Pablo Hoffman
20c8d712de
Merge pull request #1164 from scrapy/deploy-docs-2
...
deployment docs tweaks
2015-04-15 13:57:51 -03:00
Mikhail Korobov
cb5280ab0d
DOC tweak deployment docs
2015-04-15 19:25:22 +05:00
Nicolás Alejandro Ramírez Quiros
e054b3c45f
Merge pull request #1160 from scrapy/docs-theme
...
Change local docs theme
2015-04-15 10:34:19 -03:00
Mikhail Korobov
a11bd3e89d
DOC update docs readme file - sphinx >= 1.3 is now required.
2015-04-15 11:50:57 +05:00
Julia Medina
526aa07f44
Merge pull request #1156 from kmike/crawlers-accept-dicts
...
allow Crawler, CrawlerRunner and CrawlerProcess to accept dicts instead of Setting objects
2015-04-15 02:15:32 -03:00
Daniel Graña
c013baa602
Merge pull request #1161 from scrapy/telnet-disable
...
disable scrapy.telnet if twisted.conch is not available
2015-04-15 00:32:42 -03:00
Daniel Graña
e3aaeb7f06
Merge pull request #1162 from kmike/ajaxcrawl-py3
...
fix Python 3 syntax errors in ajaxcrawl.py
2015-04-15 00:32:03 -03:00
Mikhail Korobov
0b76487629
PY3 fix Python 3 syntax errors in ajaxcrawl.py
...
Tests for _has_ajaxcrawlable_meta passin Python 3.
2015-04-15 04:16:26 +05:00
Mikhail Korobov
378b6efc0e
PY3 disable scrapy.telnet if twisted.conch is not available
2015-04-15 04:06:36 +05:00
Mikhail Korobov
f40587c661
Merge pull request #1131 from torymur/master
...
Fix #1130
2015-04-15 03:52:07 +05:00
Mikhail Korobov
dd84f4bd9a
DOC remove css file which isno longer needed
2015-04-15 03:44:49 +05:00
Mikhail Korobov
7dcd7f3ca7
DOC make local docs use the same theme as readthedocs.org
...
sphinx_rtd_theme is builtin in recent Sphinx versions.
2015-04-15 03:43:30 +05:00
Mikhail Korobov
7644813286
allow Crawler, CrawlerRunner and CrawlerProcess to accept dicts instead of Setting objects
2015-04-14 23:26:05 +05:00
Nicolás Alejandro Ramírez Quiros
c2c8098562
Merge pull request #1155 from kmike/remove-deprecated
...
remove deprecated code from FeedExporter
2015-04-14 14:09:10 -03:00
Mikhail Korobov
5436ba117b
remove deprecated code from FeedExporter
2015-04-14 20:19:28 +05:00
Pablo Hoffman
71c0aface7
Merge pull request #1148 from Curita/verify-spidermanager-interface
...
Verify SPIDER_MANAGER_CLASS interface while loading it in CrawlerRunner
2015-04-13 14:25:32 -03:00
Daniel Graña
e85679fa6b
Merge pull request #1147 from Curita/pass-crawlers-to-runner-crawl-calls
...
Allow passing Crawler instances directly to CrawlerRunner.crawl()
2015-04-13 14:23:15 -03:00
Julia Medina
5af557e610
Merge pull request #1152 from jdemaeyer/fix/tests-settings-api
...
Fix deprecated settings API in tests
2015-04-12 23:56:40 -03:00
Daniel Graña
f71175a449
More replaces of references to old sel shortcut
2015-04-12 21:56:23 -03:00
Daniel Graña
f9208dbcc4
Merge pull request #1154 from dianakhuang/patch-2
...
Remove references to the `sel` object in shell.rst
2015-04-12 21:55:35 -03:00
Diana Huang
91a60d9f3d
Remove references to the sel object in shell.rst
...
The current documentation has references to the deprecated `sel` when interacting with the shell. I've removed them and replaced uses of `sel.xpath` with `response.xpath` instead.
2015-04-12 13:44:32 -04:00
Jakob de Maeyer
cf9d848f0f
Fixed deprecated settings API in cmdline test
2015-04-12 14:18:47 +02:00
Mikhail Korobov
6f000b8cd4
Merge pull request #952 from Digenis/unicode_xpath_exception
...
[MRG+1] encode unicode selector exception
2015-04-11 00:38:42 +05:00
Nikolaos-Digenis Karagiannis
bf301b7328
encode invalid xpath with unicode_escape under PY2
...
The exception quotes an xpath string
which may be unicode.
2015-04-10 22:23:40 +03:00
Julia Medina
24a07fd892
Verify SPIDER_MANAGER_CLASS interface in CrawlerRunner
2015-04-10 13:43:33 -03:00
Julia Medina
3dabde6706
Update docs for CrawlerRunner.crawl() new usage
2015-04-10 12:09:07 -03:00
Julia Medina
86b0951327
Delete crawler_deferreds doc in CrawlerRunner
...
This attribute is now an internal one since it's main use-case was
covered by CrawlerRunner.stop().
2015-04-10 12:06:27 -03:00
Julia Medina
bc70584344
Support crawlers as first positional arg in CrawlerRunner.crawl()
2015-04-10 11:52:57 -03:00
Julia Medina
f8ca5d1092
Merge pull request #1134 from nyov/remove-contrib
...
[MRG+1] dissolve contrib_exp
2015-04-10 10:33:43 -03:00
Julia Medina
543d02bc64
Avoid name clashing in deploy.py, fixes #1143
2015-04-09 21:36:24 -03:00
Pablo Hoffman
39fd2e62ef
fix typo in deploy.rst
2015-04-09 18:01:11 -03:00
Pablo Hoffman
1a12922b77
improve scrapy deploy documentation
2015-04-09 17:39:01 -03:00
Pablo Hoffman
9ea309c354
rename deployment.rst -> deploy.rst (consist with others like debug.rst)
2015-04-09 16:56:35 -03:00
Pablo Hoffman
d8184a7239
Merge pull request #1124 from rdowinton/deployment-doc
...
Added deployment section covering scrapyd-deploy and shub
2015-04-09 16:53:25 -03:00
nyov
6d48c19acd
dissolve scrapy.contrib_exp
2015-04-07 21:20:48 +00:00
Pablo Hoffman
4b11501eb5
Merge pull request #963 from tpeng/fix-xmliter-lxml
...
[MRG+1] support namespace prefix in xmliter_lxml
2015-04-07 18:13:01 -03:00
Daniel Graña
9706119336
Merge pull request #1128 from Curita/move-spider-settings-population-to-crawler
...
[MRG+1] Move spider settings population from CrawlerRunner to Crawler.__init__
2015-04-04 18:08:05 -03:00
Julia Medina
fcf97001a5
Merge pull request #1121 from nyov/nyov/py3-urllib
...
[MRG+1] more python3 compatibility changes for urllib
2015-04-02 22:33:43 -03:00
Victoria Terenina (torymur)
42f7674619
fixed unhandled error in deferred (RobotsTxtMiddleware)
2015-04-02 22:40:32 +03:00
Alexander Sibiryakov
85aa3c7596
Dns cache size and timeout options
2015-04-02 18:30:59 +02:00
Julia Medina
6f9265b44b
Move spider settings population to Crawler.__init__
2015-04-01 23:29:10 -03:00
Julia Medina
66aef0488a
Merge pull request #1127 from nyov/docs-fix
...
documentation build warning fixes
2015-04-01 17:02:28 -03:00
nyov
dc88be75e2
more python3 compatibility changes for urllib
2015-04-01 19:50:56 +00:00
nyov
92b574309e
documentation build warning fixes
2015-04-01 19:46:21 +00:00
Daniel Graña
27591b55fc
Merge pull request #1123 from sibiryakov/reactor-threadpool-size
...
[MRG+1] Reactor threadpool max size setting
2015-04-01 15:08:03 -03:00
Alexander Sibiryakov
b794cdaf4b
Broad crawls notes.
2015-04-01 12:07:03 +02:00
Alexander Sibiryakov
e7b274edf3
Reformat to 80 characters per line.
2015-04-01 11:49:55 +02:00
