Pablo Hoffman
b1d1a36a1e
add note about enclosing urls with quotes when running from command-line. closes GH-384
2013-09-18 18:01:28 -03:00
Mikhail Korobov
bb8cb41777
Merge pull request #391 from amferraz/patch-2
...
Update request-response.rst
2013-09-18 07:59:46 -07:00
cacovsky
71b320914a
Update request-response.rst
...
Fix small doc typo (too many backticks)
2013-09-18 11:45:25 -03:00
Pablo Hoffman
ee8464b471
Merge pull request #390 from kmike/mock-in-tox
...
mock is required for running tests
2013-09-17 15:51:54 -07:00
Mikhail Korobov
7ffc786fb6
mock is required for running tests since 45ff6ec28a
...
This fixes tox test running. Mock is available at Travis, but there is no harm in listing it explicitly.
2013-09-18 00:02:23 +06:00
Pablo Hoffman
2a89265b96
Merge pull request #387 from ktharmal/patch-1
...
Fixed directory location for dmoz_spider.py file
2013-09-16 11:36:13 -07:00
Kumara Tharmalingam
bbb0603091
Fixed directory location for dmoz_spider.py file
...
It should be under 'tutorial/spiders' not 'dmoz/spiders'
2013-09-15 21:55:52 -07:00
Stefan
855041ebfd
added test case for comments, see #363
2013-09-14 12:17:34 +02:00
Daniel Graña
0400b18efa
docs: list lxml as installation prerequisite
2013-09-09 12:44:26 -03:00
Stefan
6994959181
renamed to sitemap_alternate_links and added default value, see #360
2013-09-08 10:38:28 +02:00
Daniel Graña
e8be35dce5
Do not log IgnoreRequest exceptions as download failures
2013-09-08 00:24:34 -03:00
Daniel Graña
684cfc0b68
Do not silence download errors when request errback raises an exception
2013-09-08 00:19:41 -03:00
Daniel Graña
b326b87154
forms: do not submit reset inputs
2013-09-07 21:53:27 -03:00
Stefan
8ed2d0cda1
improved changes to allow retrieval of alternate links in sitemaps, see #360
2013-09-07 12:56:30 +02:00
Daniel Graña
65135db70c
increase unittest timeouts to decrease travis false positive failures
2013-09-05 21:53:50 -03:00
Daniel Graña
801564c2c9
exporters linting
2013-09-05 17:00:16 -03:00
Daniel Graña
2be3f7132a
Do not fail on when json exporter are instanciated with extra keywords. closes #379
2013-09-05 16:57:27 -03:00
Daniel Graña
92d14d4a03
Fix permission and set umask before generating sdist tarball
2013-09-03 23:22:17 -03:00
Daniel Graña
e6b3ca0180
Add 0.18.2 release notes
2013-09-03 14:29:40 -03:00
Daniel Graña
9b821df482
Merge pull request #330 from alexcepoi/contracts_fix
2013-09-03 11:41:04 -03:00
Daniel Graña
f54d5c5896
Merge pull request #370 from loucash/master
...
FilesPipeline extracted and separated from ImagesPipeline
2013-09-03 07:29:30 -07:00
Alex Cepoi
50b653e5ae
don't start reactor if no crawler is scheduled
2013-09-02 18:43:25 +02:00
Lukasz Biedrycki
0a8bf2c9e4
after additional tests with actual s3: k.set_contents_from_string is working
2013-08-28 18:00:54 +02:00
Daniel Graña
0f00b1602a
merge 0.18 release notes
2013-08-27 18:47:46 -03:00
Daniel Graña
ac218ed418
no need to test latest versions of dependencies on python 2.6
2013-08-27 17:38:28 -03:00
Daniel Graña
c5222b4b8a
remove phanton build row from travis matrix due to travis-ci/travis-core#1027
2013-08-27 17:19:45 -03:00
Daniel Graña
ad140a2763
another try to limit travis build matrix
2013-08-27 16:53:54 -03:00
Daniel Graña
7394969236
test pypy with travis
2013-08-27 15:31:08 -03:00
Daniel Graña
401e888ced
test lucid and precise python packages under their respective python versions
2013-08-27 15:30:10 -03:00
Daniel Graña
588a262b73
fix crawling tests under twisted pre 11.0.0
2013-08-27 14:05:37 -03:00
Daniel Graña
92826586d5
py26 can not format zero length fields {}
2013-08-27 12:12:50 -03:00
Daniel Graña
caa0f90263
test PotentiaDataLoss errors on unbound responses
2013-08-27 11:53:18 -03:00
Daniel Graña
6720acc1af
Treat responses without content-length or Transfer-Encoding as good responses
...
There is not a way to determine if responses without Content-Length or Transfer-Encoding are complete, this change treat them as good responses but flags them as "partial".
This is backout change only for this functionality of 3c64a989
2013-08-27 11:10:20 -03:00
Pablo Hoffman
da1f6d3144
remove unused imports and some assorted pylint-ing
2013-08-23 13:03:34 -03:00
Pablo Hoffman
19ff9ac4f9
url/body attributes of Request/Response objects are now immutable
2013-08-23 12:43:22 -03:00
Pablo Hoffman
86230c0ab8
added quantal & raring to support ubuntu releases
2013-08-22 21:49:55 -03:00
Lukasz Biedrycki
83bd151c57
backwards compatibility: image_key & image_downloaded
2013-08-22 15:13:59 +02:00
Lukasz Biedrycki
2b11e7da93
backwards compatibility: image_key
2013-08-22 14:44:04 +02:00
Lukasz Biedrycki
02d08722e7
typo
2013-08-22 14:33:28 +02:00
Lukasz Biedrycki
4c89d8b298
remarks
2013-08-22 12:26:56 +02:00
Daniel Graña
efb4f32d80
do no include ResponseFailed if http11 handler is not enabled
2013-08-21 19:04:12 -03:00
Daniel Graña
3c64a989ca
New HTTP client wraps connection losts in ResponseFailed exception. fix #373
2013-08-21 17:28:26 -03:00
Pablo Hoffman
071172cbd6
fix retry middleware which didn't retry certain connection errors after the upgrade to http1 client, closes GH-373
2013-08-21 14:32:33 -03:00
Pablo Hoffman
6e4ed8b004
Merge pull request #372 from kmike/xml-exporter-fix-for-2.7.4
...
fix XmlItemExporter in Python 2.7.4 and 2.7.5
2013-08-21 09:55:51 -07:00
Mikhail Korobov
a6e6ca06b9
fix XmlItemExporter in Python 2.7.4 and 2.7.5
2013-08-21 06:05:40 +06:00
Alex Cepoi
9c092536c5
remove redundant kwarg
2013-08-21 01:52:36 +02:00
Alex Cepoi
7bdeabc0ed
fix log intermixing in scrapy shell command
2013-08-21 01:42:03 +02:00
Alex Cepoi
902208ca58
fix some missing logs with new crawlerprocess
2013-08-20 14:02:55 +02:00
Lukasz Biedrycki
96077ccf48
typo
2013-08-20 11:29:55 +02:00
Alex Cepoi
a812def59e
port all scrapy commands to new CrawlerProcess
2013-08-19 23:40:23 +02:00
