Pablo Hoffman
fe9ae1d46a
Merge pull request #406 from nopper/master
...
Handling inconsistencies among Twisted releases
2013-10-01 09:59:00 -07:00
Francesco Piccinno
cc8a93f185
Handling inconsistencies among Twisted releases
2013-10-01 14:36:44 +02:00
Pablo Hoffman
56275b1b9c
Merge pull request #404 from darkrho/djangoitem-docs
...
Add a section to DjangoItem docs page regarding setting up Django's settings.
2013-09-30 11:15:32 -07:00
Rolando Espinoza
d6e3eae527
docs: added section regarding setting up django's settings.
2013-09-30 09:58:10 -04:00
Rolando Espinoza
0cc1d870db
docs: minor tidy up sample code and missing shell prompts.
2013-09-30 09:58:10 -04:00
Mikhail Korobov
36cb54d302
Merge pull request #402 from LorenDavie/master
...
Corrected typo.
2013-09-29 14:21:04 -07:00
Loren Davie
8af0e89e85
Corrected typo.
2013-09-29 17:06:46 -04:00
Pablo Hoffman
1a90ee51c0
Merge pull request #401 from LorenDavie/master
...
Added dynamic creation of item classes to practices.rst. Issue
2013-09-29 07:20:52 -07:00
Loren Davie
f49f5724d5
Added dynamic creation of item classes to practices.rst.
2013-09-28 09:00:48 -04:00
Pablo Hoffman
780ef128d2
Merge pull request #399 from irgmedeiros/patch-1
...
Update practices.rst
2013-09-27 14:03:51 -07:00
irgmedeiros
d9e0fdc9aa
Update practices.rst
...
With this modification scrapy runs the spider with project settings. The previous example ran only with default settings resulting in ignoring all user settings as pipelines for example.
2013-09-27 17:56:30 -03:00
Daniel Graña
265910aae6
Merge pull request #363 from taikano/sitemap_alternate
...
also fetch alternate URLs from sitemaps, see #360
2013-09-26 09:15:02 -07:00
Pablo Hoffman
12280c2a95
fix sphinx references in doc
2013-09-25 15:13:17 -03:00
Mikhail Korobov
b74e5aa551
fix ITEM_PIPELINES setting handling
...
* item_pipelines variable was unused so the fallback didn't work;
* added support for fallback in case of ITEM_PIPELINIES defined as set.
2013-09-25 20:37:06 +06:00
Daniel Graña
61e89d8247
Merge pull request #393 from scrapy/itempipe-dict
...
Make ITEM_PIPELINE setting a dict
2013-09-23 14:12:26 -07:00
Pablo Hoffman
fc388f4636
Make ITEM_PIPELINE setting a dict
...
This is for consistency with how spider and downloader middlewares are
defined. ITEM_PIPELINE_BASE was also added and both remain empty.
Backwards compatibility is kept (with a warning) with list-based
ITEM_PIPELINES.
2013-09-23 17:50:43 -03:00
Pablo Hoffman
b1d1a36a1e
add note about enclosing urls with quotes when running from command-line. closes GH-384
2013-09-18 18:01:28 -03:00
Mikhail Korobov
bb8cb41777
Merge pull request #391 from amferraz/patch-2
...
Update request-response.rst
2013-09-18 07:59:46 -07:00
cacovsky
71b320914a
Update request-response.rst
...
Fix small doc typo (too many backticks)
2013-09-18 11:45:25 -03:00
Pablo Hoffman
ee8464b471
Merge pull request #390 from kmike/mock-in-tox
...
mock is required for running tests
2013-09-17 15:51:54 -07:00
Mikhail Korobov
7ffc786fb6
mock is required for running tests since 45ff6ec28a
...
This fixes tox test running. Mock is available at Travis, but there is no harm in listing it explicitly.
2013-09-18 00:02:23 +06:00
Pablo Hoffman
2a89265b96
Merge pull request #387 from ktharmal/patch-1
...
Fixed directory location for dmoz_spider.py file
2013-09-16 11:36:13 -07:00
Kumara Tharmalingam
bbb0603091
Fixed directory location for dmoz_spider.py file
...
It should be under 'tutorial/spiders' not 'dmoz/spiders'
2013-09-15 21:55:52 -07:00
Stefan
855041ebfd
added test case for comments, see #363
2013-09-14 12:17:34 +02:00
Daniel Graña
0400b18efa
docs: list lxml as installation prerequisite
2013-09-09 12:44:26 -03:00
Stefan
6994959181
renamed to sitemap_alternate_links and added default value, see #360
2013-09-08 10:38:28 +02:00
Daniel Graña
b326b87154
forms: do not submit reset inputs
2013-09-07 21:53:27 -03:00
Stefan
8ed2d0cda1
improved changes to allow retrieval of alternate links in sitemaps, see #360
2013-09-07 12:56:30 +02:00
Daniel Graña
65135db70c
increase unittest timeouts to decrease travis false positive failures
2013-09-05 21:53:50 -03:00
Daniel Graña
801564c2c9
exporters linting
2013-09-05 17:00:16 -03:00
Daniel Graña
2be3f7132a
Do not fail on when json exporter are instanciated with extra keywords. closes #379
2013-09-05 16:57:27 -03:00
Daniel Graña
92d14d4a03
Fix permission and set umask before generating sdist tarball
2013-09-03 23:22:17 -03:00
Daniel Graña
e6b3ca0180
Add 0.18.2 release notes
2013-09-03 14:29:40 -03:00
Daniel Graña
9b821df482
Merge pull request #330 from alexcepoi/contracts_fix
2013-09-03 11:41:04 -03:00
Daniel Graña
f54d5c5896
Merge pull request #370 from loucash/master
...
FilesPipeline extracted and separated from ImagesPipeline
2013-09-03 07:29:30 -07:00
Alex Cepoi
50b653e5ae
don't start reactor if no crawler is scheduled
2013-09-02 18:43:25 +02:00
Lukasz Biedrycki
0a8bf2c9e4
after additional tests with actual s3: k.set_contents_from_string is working
2013-08-28 18:00:54 +02:00
Daniel Graña
0f00b1602a
merge 0.18 release notes
2013-08-27 18:47:46 -03:00
Daniel Graña
ac218ed418
no need to test latest versions of dependencies on python 2.6
2013-08-27 17:38:28 -03:00
Daniel Graña
c5222b4b8a
remove phanton build row from travis matrix due to travis-ci/travis-core#1027
2013-08-27 17:19:45 -03:00
Daniel Graña
ad140a2763
another try to limit travis build matrix
2013-08-27 16:53:54 -03:00
Daniel Graña
7394969236
test pypy with travis
2013-08-27 15:31:08 -03:00
Daniel Graña
401e888ced
test lucid and precise python packages under their respective python versions
2013-08-27 15:30:10 -03:00
Daniel Graña
588a262b73
fix crawling tests under twisted pre 11.0.0
2013-08-27 14:05:37 -03:00
Daniel Graña
92826586d5
py26 can not format zero length fields {}
2013-08-27 12:12:50 -03:00
Daniel Graña
caa0f90263
test PotentiaDataLoss errors on unbound responses
2013-08-27 11:53:18 -03:00
Daniel Graña
6720acc1af
Treat responses without content-length or Transfer-Encoding as good responses
...
There is not a way to determine if responses without Content-Length or Transfer-Encoding are complete, this change treat them as good responses but flags them as "partial".
This is backout change only for this functionality of 3c64a989
2013-08-27 11:10:20 -03:00
Pablo Hoffman
da1f6d3144
remove unused imports and some assorted pylint-ing
2013-08-23 13:03:34 -03:00
Pablo Hoffman
19ff9ac4f9
url/body attributes of Request/Response objects are now immutable
2013-08-23 12:43:22 -03:00
Pablo Hoffman
86230c0ab8
added quantal & raring to support ubuntu releases
2013-08-22 21:49:55 -03:00
