Daniel Graña
801564c2c9
exporters linting
2013-09-05 17:00:16 -03:00
Daniel Graña
2be3f7132a
Do not fail on when json exporter are instanciated with extra keywords. closes #379
2013-09-05 16:57:27 -03:00
Daniel Graña
92d14d4a03
Fix permission and set umask before generating sdist tarball
2013-09-03 23:22:17 -03:00
Daniel Graña
e6b3ca0180
Add 0.18.2 release notes
2013-09-03 14:29:40 -03:00
Daniel Graña
9b821df482
Merge pull request #330 from alexcepoi/contracts_fix
2013-09-03 11:41:04 -03:00
Daniel Graña
f54d5c5896
Merge pull request #370 from loucash/master
...
FilesPipeline extracted and separated from ImagesPipeline
2013-09-03 07:29:30 -07:00
Alex Cepoi
50b653e5ae
don't start reactor if no crawler is scheduled
2013-09-02 18:43:25 +02:00
Lukasz Biedrycki
0a8bf2c9e4
after additional tests with actual s3: k.set_contents_from_string is working
2013-08-28 18:00:54 +02:00
Daniel Graña
0f00b1602a
merge 0.18 release notes
2013-08-27 18:47:46 -03:00
Daniel Graña
ac218ed418
no need to test latest versions of dependencies on python 2.6
2013-08-27 17:38:28 -03:00
Daniel Graña
c5222b4b8a
remove phanton build row from travis matrix due to travis-ci/travis-core#1027
2013-08-27 17:19:45 -03:00
Daniel Graña
ad140a2763
another try to limit travis build matrix
2013-08-27 16:53:54 -03:00
Daniel Graña
7394969236
test pypy with travis
2013-08-27 15:31:08 -03:00
Daniel Graña
401e888ced
test lucid and precise python packages under their respective python versions
2013-08-27 15:30:10 -03:00
Daniel Graña
588a262b73
fix crawling tests under twisted pre 11.0.0
2013-08-27 14:05:37 -03:00
Daniel Graña
92826586d5
py26 can not format zero length fields {}
2013-08-27 12:12:50 -03:00
Daniel Graña
caa0f90263
test PotentiaDataLoss errors on unbound responses
2013-08-27 11:53:18 -03:00
Daniel Graña
6720acc1af
Treat responses without content-length or Transfer-Encoding as good responses
...
There is not a way to determine if responses without Content-Length or Transfer-Encoding are complete, this change treat them as good responses but flags them as "partial".
This is backout change only for this functionality of 3c64a989
2013-08-27 11:10:20 -03:00
Pablo Hoffman
da1f6d3144
remove unused imports and some assorted pylint-ing
2013-08-23 13:03:34 -03:00
Pablo Hoffman
19ff9ac4f9
url/body attributes of Request/Response objects are now immutable
2013-08-23 12:43:22 -03:00
Pablo Hoffman
86230c0ab8
added quantal & raring to support ubuntu releases
2013-08-22 21:49:55 -03:00
Lukasz Biedrycki
83bd151c57
backwards compatibility: image_key & image_downloaded
2013-08-22 15:13:59 +02:00
Lukasz Biedrycki
2b11e7da93
backwards compatibility: image_key
2013-08-22 14:44:04 +02:00
Lukasz Biedrycki
02d08722e7
typo
2013-08-22 14:33:28 +02:00
Lukasz Biedrycki
4c89d8b298
remarks
2013-08-22 12:26:56 +02:00
Daniel Graña
efb4f32d80
do no include ResponseFailed if http11 handler is not enabled
2013-08-21 19:04:12 -03:00
Daniel Graña
3c64a989ca
New HTTP client wraps connection losts in ResponseFailed exception. fix #373
2013-08-21 17:28:26 -03:00
Pablo Hoffman
071172cbd6
fix retry middleware which didn't retry certain connection errors after the upgrade to http1 client, closes GH-373
2013-08-21 14:32:33 -03:00
Pablo Hoffman
6e4ed8b004
Merge pull request #372 from kmike/xml-exporter-fix-for-2.7.4
...
fix XmlItemExporter in Python 2.7.4 and 2.7.5
2013-08-21 09:55:51 -07:00
Mikhail Korobov
a6e6ca06b9
fix XmlItemExporter in Python 2.7.4 and 2.7.5
2013-08-21 06:05:40 +06:00
Alex Cepoi
9c092536c5
remove redundant kwarg
2013-08-21 01:52:36 +02:00
Alex Cepoi
7bdeabc0ed
fix log intermixing in scrapy shell command
2013-08-21 01:42:03 +02:00
Alex Cepoi
902208ca58
fix some missing logs with new crawlerprocess
2013-08-20 14:02:55 +02:00
Lukasz Biedrycki
96077ccf48
typo
2013-08-20 11:29:55 +02:00
Alex Cepoi
a812def59e
port all scrapy commands to new CrawlerProcess
2013-08-19 23:40:23 +02:00
Alex Cepoi
a9a911fb09
default to multi crawler commands
...
compatibility for old single crawler process
deprecated command's `crawler` property
logs about settings only shown once
2013-08-19 22:43:29 +02:00
Mikhail Korobov
034ffae60f
Recommend Pillow instead of PIL. Closes GH-317.
2013-08-18 00:44:01 +06:00
Lukasz Biedrycki
45ff6ec28a
Test reorganization and new tests for Files and Images Pipelines, PEP8 changes in MediaPipeline
2013-08-16 18:47:56 +02:00
Lukasz Biedrycki
76ce8c5238
FilesPipeline which enalbes to download any files.
...
It has been extracted from ImagesPipelines.
ImagesPipeline is built on top of FilesPipeline and consist only with convert image and thumbnail generation logic.
2013-08-16 17:02:31 +02:00
Pablo Hoffman
f95b164d06
Merge pull request #367 from berendiwema/master
...
#327 - Support STARTTLS / SSL option in email sender
2013-08-14 07:17:30 -07:00
Berend Iwema
32b6364bcd
#327 - Support STARTTLS / SSL option in email sender
2013-08-14 12:59:01 +02:00
Pablo Hoffman
49952a45ae
add docstring to PythonItemExporter
2013-08-14 02:49:14 -03:00
Pablo Hoffman
c0b26e3d49
minor updates to 0.18 release notes
2013-08-14 01:39:44 -03:00
Daniel Graña
c6157d37e9
Merge pull request #366 from kalessin/exporter
...
added a python native classes item exporter
2013-08-13 06:55:27 -07:00
olveyra
456b6f2ef5
added a python native classes item exporter
2013-08-13 13:48:28 +00:00
Daniel Graña
b43b5f575e
adjust http11 pool size to per-domain concurrency
2013-08-12 22:43:40 -03:00
Pablo Hoffman
892386ee01
tox.ini: disable sitepackages on windows, as a compiler is often not available
2013-08-12 18:59:41 -03:00
Daniel Graña
ed5b9068d2
fix contributters list format
2013-08-12 11:26:16 -03:00
Daniel Graña
80e25c5980
bumped version to 0.19.0
2013-08-09 19:09:47 -03:00
Daniel Graña
a6693c9a5c
updated release notes and bumped version to 0.18.0
2013-08-09 19:02:28 -03:00
