Cameron Lane
eefa878f98
[ #744 ] Ensure domain is not None before building regex
2014-06-12 11:42:14 -04:00
Pablo Hoffman
ad11fc01cb
crate release notes for 0.24 and #699 to it
2014-06-10 18:46:53 -03:00
Pablo Hoffman
9a4ee20991
Merge pull request #742 from dangra/sort-scrapy-list
...
sort spiders in "scrapy list" cmd.
2014-06-03 17:34:04 -03:00
Daniel Graña
f6d48213f6
sort spiders in "scrapy list" cmd. closes #736
2014-06-03 15:36:58 -03:00
Daniel Graña
1c9effd7b4
Avoid IPython warning. thanks @bryant1410. closes #623
2014-06-03 15:11:20 -03:00
Daniel Graña
9830efb946
Merge pull request #697 from allait/master
...
Add a note about reporting security issues
2014-06-03 11:13:13 -03:00
Alex Cepoi
02ac973a61
fix contracts tests
2014-06-03 10:43:08 -03:00
Daniel Graña
7752074c51
Merge pull request #719 from Digenis/master
...
document spider.closed() shortcut
2014-06-03 10:01:15 -03:00
Daniel Graña
791889f51d
Merge pull request #733 from alexcepoi/contracts_tweaks
...
improvements to scrapy check/contracts
2014-06-03 09:55:48 -03:00
Alex Cepoi
7130df2d96
improvements to scrapy check/contracts
...
* report number of contracts run and time spent (similar to unittest)
* exit with code 1 if there are failures (similar to unittest)
* capture and report errback errors (twisted or mw errors)
* capture and report callback errors (exceptions raised by spider)
2014-06-03 14:51:25 +02:00
Daniel Graña
0467ad7719
Merge pull request #699 from allait/telnet-port
...
Bind telnet console and webservice to 127.0.0.1 by default
2014-06-03 09:51:16 -03:00
Daniel Graña
8a1c67fdaa
Merge pull request #724 from rafallo/master
...
item_dropped signal should pass response arg as item_scraped does
2014-06-03 09:48:06 -03:00
Daniel Graña
21df09d809
Merge pull request #726 from nyov/fix
...
some cosmetics
2014-06-03 09:44:09 -03:00
Daniel Graña
2a42c1c62e
Merge pull request #738 from chekunkov/downloader_setting
...
Downloader setting
2014-06-03 09:36:53 -03:00
Daniel Graña
597fac25ec
force installation of w3lib and queuelib for trunk env
2014-06-02 20:29:35 +00:00
Daniel Graña
2ba78e715b
fix tests after changes introduced by scrapy/w3lib#21
2014-06-02 20:29:12 +00:00
Daniel Graña
65f69e1609
limit Twisted support to pre-14.0.0 while #718 is fixed
2014-06-02 13:27:20 -03:00
Alexander Chekunkov
fa5a67729b
DOWNLOADER setting
2014-06-02 13:11:02 +03:00
Alexander Chekunkov
1fba64d34e
DOWNLOADER setting
2014-06-02 13:05:22 +03:00
Daniel Graña
156d347c00
Merge pull request #727 from tpeng/fix-scrapy-check-exitcode
...
set the exit code to non-zero when contracts fails
2014-05-21 09:50:03 -04:00
tpeng
b2ca3cf7e6
set the exit code to non-zero when contracts fails
2014-05-21 09:31:51 +02:00
nyov
1cd21f97e0
update a link reference
2014-05-20 05:19:12 +00:00
nyov
2899a778e6
better call to parent class
2014-05-20 04:56:57 +00:00
Rafal Jagoda
ad6e449c63
Merge remote-tracking branch 'upstream/master'
2014-05-19 16:18:33 +01:00
Nikolaos-Digenis Karagiannis
24073a2624
document spider.closed() shortcut
2014-05-12 22:25:18 +03:00
Pablo Hoffman
2bf09b8a20
Merge pull request #714 from andrewbaxter/master
...
Elaborated Request priority value in Scrapy docs
2014-05-07 17:53:38 -04:00
Rendaw
8bdb6e2e3e
Elaborated request priority value.
2014-05-07 19:14:45 +09:00
Pablo Hoffman
6431e70dbe
Merge pull request #711 from cerivera/master
...
grammatical issue
2014-05-06 16:59:04 -04:00
Carlos Rivera
946b854ddf
grammatical issue
2014-05-06 15:41:59 -05:00
Pablo Hoffman
c424f3709c
add Julia to SEP-019 authors
2014-05-04 12:02:35 -03:00
Daniel Graña
4bd8eb17a8
Merge pull request #705 from Curita/sep19-update
...
Per-spider settings and api cleanup: sep#19 update
2014-05-04 10:13:33 -04:00
Rafal Jagoda
9d226e6266
add response arg to item_dropped signal handlers
...
#710
2014-05-02 14:42:01 +01:00
Daniel Graña
197e360730
Merge pull request #707 from redapple/gh-issue-706-start-requests
...
[MRG] Fix engine to support filtered start_requests
2014-04-28 12:13:40 -04:00
Paul Tremberth
7199555e1a
Check pending start_requests before calling _spider_idle() in engine ( fixes #706 )
2014-04-28 16:57:33 +02:00
Daniel Graña
ad14cf058b
Merge pull request #597 from chekunkov/dupefilter_request_fingerprint_test_and_docs
...
Dupefilter request_fingerprint test and docs
2014-04-28 10:37:35 -04:00
Alexander Chekunkov
4aa6179af7
added short RFPDupeFilter.request_fingerprint interface description
2014-04-26 15:46:53 +03:00
Alexander Chekunkov
baaa077725
added note about RFPDupeFilter.request_fingerprint overriding to the settings documentation
2014-04-26 15:43:12 +03:00
Alexander Chekunkov
6e4c77c684
test for RFPDupeFilter.request_fingerprint overriding
2014-04-26 15:42:01 +03:00
Paul Tremberth
4898d96337
Add tests for start requests, filtered and non-filtered
2014-04-26 13:47:42 +02:00
Pablo Hoffman
3932877bcd
signals doc: make argument order more consistent with code (although it doesn't matter in practice)
2014-04-25 16:03:32 -03:00
Julia Medina
9cbce9105b
sep#19 proposed changes
2014-04-25 01:57:24 -03:00
Pablo Hoffman
eb07e09166
Merge pull request #663 from pawl/patch-1
...
fixed typo
2014-04-24 17:59:36 -04:00
Mikhail Korobov
bdea071af3
DOC selectors.rst cleanup
2014-04-24 22:54:17 +06:00
ncp1113
5f18816428
for loops have to have a : at the end of the line
...
changed 3 instances
2014-04-24 22:50:21 +06:00
Mikhail Korobov
e8f56af9e1
Merge pull request #698 from allait/pep8templates
...
Fix PEP8 warnings in project and spider template files
2014-04-24 22:43:55 +06:00
Alexey Bezhan
5d37e4dc5a
Fix PEP8 warnings in spider templates
2014-04-24 17:30:35 +01:00
Alexey Bezhan
1d6cc257ea
Fix PEP8 warnings in project template files
2014-04-24 17:30:13 +01:00
Daniel Graña
9acf2be475
Merge pull request #690 from dangra/kmike-response-selectors
...
Added docs response selector shortcuts
2014-04-24 12:11:39 -04:00
Daniel Graña
1e7ddc8e52
Restore and deprecate "sel" shortcut
2014-04-24 12:47:45 -03:00
Daniel Graña
18412d75b5
Ammend example nesting selectors
2014-04-24 10:32:17 -03:00
