Mikhail Korobov
ef81127b99
Merge pull request #760 from dangra/546-remaining
...
recognize jl extension as jsonlines exporter and update docs
2014-06-25 23:31:48 +06:00
Daniel Graña
5b2faf61c3
recognize jl extension as jsonlines exporter and update docs
2014-06-25 13:55:15 -03:00
Daniel Graña
f98bf2c521
Merge pull request #737 from Curita/settings-cleanup
...
Settings api cleanup
2014-06-25 13:01:50 -03:00
Daniel Graña
f224ac1328
Restore compatibility with Settings.overrides while still deprecating it
2014-06-25 13:01:12 -03:00
Daniel Graña
f72b1597b4
Merge pull request #718 from dangra/twisted-14.0.0
...
[WIP] does it work with Twisted 14.0.0?
2014-06-25 09:50:31 -03:00
Daniel Graña
d48158427a
hold a reference to backwards compatible _contextFactory
2014-06-25 04:22:18 -03:00
Daniel Graña
bfe02b019e
Revert "limit Twisted support to pre-14.0.0 while #718 is fixed"
...
This reverts commit 65f69e16095799e05de79a3f8f449f7b9edb27aa.
2014-06-25 04:22:18 -03:00
Daniel Graña
559af943dc
Add bumpversion config
2014-06-25 03:19:04 -03:00
Daniel Graña
1b32ece918
Merge branch 'settings-cleanup' of https://github.com/Curita/scrapy into Curita-settings-cleanup
2014-06-25 02:55:20 -03:00
Daniel Graña
4300334a23
Upload sdist and wheel packages to pypi using travis-ci deploys
2014-06-25 02:53:12 -03:00
Daniel Graña
499438ec44
indent parsed-literal as part of ordered list
2014-06-24 10:32:56 -03:00
Pablo Hoffman
ee5c3209af
doc: use |version| substitution in ubuntu packages
2014-06-24 09:52:51 -03:00
Paul Tremberth
90e69141d3
Add doc on LxmlLinkExtractor class
2014-06-23 22:35:56 +02:00
Paul Tremberth
d34a57a10e
Add LxmlLinkExtractor class similar to SgmlLinkExtractor ( #528 )
2014-06-23 22:35:56 +02:00
Daniel Graña
05ffca2781
Merge pull request #626 from dangra/500-leveldb-cache
...
[WIP] Add a LevelDB cache backend
2014-06-23 15:30:13 -03:00
Daniel Graña
a90fd81c97
add leveldb cache backend docs
2014-06-23 14:37:29 -03:00
Daniel Graña
9d57ecfc35
Merge pull request #752 from tpeng/slightly-improve-contract-tests
...
print spider name even it has no contract tests when -v is specified
2014-06-18 03:15:21 -03:00
Daniel Graña
7449b25bed
Merge pull request #745 from crlane/cl/offsite-middleware-bugfix
...
Cl/offsite middleware bugfix
2014-06-18 03:01:21 -03:00
tpeng
5fb22ba4e7
print spider name even it has no contract tests when -v is specified
2014-06-17 13:39:10 +02:00
Pablo Hoffman
7fd8a1a387
minor change to request_scheduled signal doc
2014-06-14 20:51:16 -03:00
Pablo Hoffman
313ff26a79
Merge pull request #746 from Digenis/doc
...
Document signal "request_scheduled"
2014-06-14 20:49:43 -03:00
Nikolaos-Digenis Karagiannis
5f79940295
Document signal "request_scheduled"
2014-06-14 11:08:18 +03:00
Cameron Lane
a069d36652
[ #744 ] Test for allowed domains including NoneTypes
2014-06-12 11:43:03 -04:00
Cameron Lane
eefa878f98
[ #744 ] Ensure domain is not None before building regex
2014-06-12 11:42:14 -04:00
Pablo Hoffman
ad11fc01cb
crate release notes for 0.24 and #699 to it
2014-06-10 18:46:53 -03:00
Julia Medina
6730e3c9f7
Fix settings usage across tests
2014-06-10 11:26:50 -03:00
Julia Medina
43ae69e436
Fix settings usage in runspider and crawl commands
...
Explicit conversion of FEED_EXPORTERS and FEED_EXPORTERS_BASE values into
dictionaries, and cast of the keys to a list for py3 support.
2014-06-10 11:26:50 -03:00
Julia Medina
c81b087de1
Change how settings are overriden in ScrapyCommands
2014-06-10 11:26:50 -03:00
Julia Medina
080a28f3e9
Change command's default_settings population in cmdline.py
2014-06-10 11:26:50 -03:00
Julia Medina
c496468e2a
get_project_settings now returns a Settings instance
2014-06-10 11:26:50 -03:00
Julia Medina
b397f5e935
Update get_crawler method in utils/test.py with new Settings interface
...
This early adaptation of the new Settings interface is needed for running tests
2014-06-10 11:26:50 -03:00
Julia Medina
ed033854e2
setmodule helper method on Settings class
2014-06-10 11:26:50 -03:00
Julia Medina
bdca06240c
Fix settings repr on the logs of the shell and tutorial docs topics
2014-06-10 11:26:50 -03:00
Julia Medina
8a3a6236d9
Settings topic updated
2014-06-10 11:26:50 -03:00
Julia Medina
00e6470ed0
Fix and extend the documentation of the new Settings api
2014-06-10 11:26:38 -03:00
Julia Medina
b4b17049e9
Settings and SettingsAtribute tests
2014-06-10 11:21:44 -03:00
Julia Medina
22572834fb
Deprecate CrawlerSettings, as its functionality is replicable by Settings class
2014-06-10 11:19:15 -03:00
Julia Medina
95ba68683e
New set and setdict method using SettingsAttribute in Settings
2014-06-10 11:19:15 -03:00
Julia Medina
384efce0a6
Settings priorities dictionary
2014-06-10 11:12:44 -03:00
Julia Medina
1276b12c25
New SettingsAttribute class
2014-06-09 19:08:08 -03:00
Daniel Graña
752787e634
Add a LevelDB cache backend
2014-06-04 16:09:07 -03:00
Pablo Hoffman
9a4ee20991
Merge pull request #742 from dangra/sort-scrapy-list
...
sort spiders in "scrapy list" cmd.
2014-06-03 17:34:04 -03:00
Daniel Graña
f6d48213f6
sort spiders in "scrapy list" cmd. closes #736
2014-06-03 15:36:58 -03:00
Daniel Graña
1c9effd7b4
Avoid IPython warning. thanks @bryant1410. closes #623
2014-06-03 15:11:20 -03:00
Daniel Graña
9830efb946
Merge pull request #697 from allait/master
...
Add a note about reporting security issues
2014-06-03 11:13:13 -03:00
Alex Cepoi
02ac973a61
fix contracts tests
2014-06-03 10:43:08 -03:00
Daniel Graña
7752074c51
Merge pull request #719 from Digenis/master
...
document spider.closed() shortcut
2014-06-03 10:01:15 -03:00
Daniel Graña
791889f51d
Merge pull request #733 from alexcepoi/contracts_tweaks
...
improvements to scrapy check/contracts
2014-06-03 09:55:48 -03:00
Alex Cepoi
7130df2d96
improvements to scrapy check/contracts
...
* report number of contracts run and time spent (similar to unittest)
* exit with code 1 if there are failures (similar to unittest)
* capture and report errback errors (twisted or mw errors)
* capture and report callback errors (exceptions raised by spider)
2014-06-03 14:51:25 +02:00
Daniel Graña
0467ad7719
Merge pull request #699 from allait/telnet-port
...
Bind telnet console and webservice to 127.0.0.1 by default
2014-06-03 09:51:16 -03:00
