Paul Tremberth
03ff19d188
Update docs for new "referrer_policy" Request.meta key
2017-03-01 17:51:23 +01:00
Mikhail Korobov
7b49b9c0f5
Merge pull request #2590 from rolando-contrib/handle-data-loss-gracefully
...
[MRG+2] Handle data loss gracefully.
2017-03-01 20:23:19 +05:00
Rolando Espinoza
f01ae6ffcd
Handle data loss gracefully.
...
Websites that return a wrong ``Content-Length`` header may cause a data
loss error. Also when a chunked response is not finished properly.
This change adds a new setting ``DOWNLOAD_FAIL_ON_DATALOSS`` (default:
``True``) and request.meta key ``download_fail_on_dataloss``.
2017-03-01 11:43:53 -03:00
MikeinRealLife
441f25507e
fixed typo
...
removed duplicate line
2017-02-22 21:23:27 -08:00
MikeinRealLife
96a570a93a
fixed ticket #2574
2017-02-22 21:17:34 -08:00
Omer Schleifer
ff3e299eb0
[MRG+2] add flags to request ( #2082 )
...
* add flags to request
* fxi test - add flags to request
* fix test(2) - add flags to request
* fix test(2) - add flags to request
* Updated test to reqser with flags field of request
* Updated documntation with flags field of request
* fix test identation
* fix test failed
* make the change backward comptaible
* remove unrequired spaces, fix documentation request flags
* remove unrequired space
* fx assert equal
* flags default is empty list
* Add flags to request
* add flags to request
* fxi test - add flags to request
* fix test(2) - add flags to request
* fix test(2) - add flags to request
* Updated test to reqser with flags field of request
* Updated documntation with flags field of request
* fix test identation
* fix test failed
* make the change backward comptaible
* remove unrequired spaces, fix documentation request flags
* remove unrequired space
* fx assert equal
* flags default is empty list
* add flags to request squashed commits
2017-02-20 20:42:29 +06:00
Daniel Graña
c68140e68a
Merge pull request #2540 from scrapy/response-follow
...
response.follow
2017-02-20 11:21:21 -03:00
Ashish Kulkarni
165e2cb8c9
document issue with FormRequest.from_response due to bug in lxml
...
This can make the spider fail due to incorrect values being posted
server-side, which is extremely hard to debug because it is easy
to miss leading/trailing whitespace, even with a logging proxy.
The fix was merged for lxml 3.8 in lxml/lxml#228 so document that
as well.
2017-02-17 14:54:22 +05:30
Mikhail Korobov
5b79c6a679
DOC document response.follow methods; expand the tutorial
2017-02-16 00:06:52 +05:00
Mikhail Korobov
877057fac0
initial response.follow implementation
2017-02-15 01:22:53 +05:00
Paul Tremberth
7d0b89042f
Merge pull request #2533 from djrobust/patch-1
...
[MRG+1] Use yield with nested parsing of responses
2017-02-08 13:02:50 +01:00
djrobust
3021084f37
Use 'yield' when parsing multiple responses
...
Use 'yield' consistently across examples of parse functions.
2017-02-04 20:07:05 -08:00
Raul Gallegos
df1a42419f
adding formid to FormRequest documentation
2017-01-14 20:45:20 -05:00
Mikhail Korobov
570e12b5db
Merge pull request #2328 from scrapy/download-latency-meta-docs
...
[MRG+1] Document `download_latency` meta key
2016-11-14 21:24:14 +05:00
Valdir Stumm Junior
7025d6656a
document download_latency meta key
2016-11-14 13:06:18 -02:00
bopace
fd016ee71b
Fixed wording of documentation
2016-10-18 09:37:45 -06:00
Bo Pace
bfe28ae707
Added documentation about accessing header values
2016-10-17 14:10:05 -06:00
Thom Dixon
f68dc3026d
Fix indentation
2016-08-24 09:11:27 -07:00
Thom Dixon
633abfbea1
Correct documentation about Response parameters
...
This fixes issue #2196
2016-08-24 08:47:52 -07:00
Paul Tremberth
b3367c7acd
DOC Add info and example on errbacks
2016-05-18 18:00:09 +02:00
Aron Bordin
2cfe9e424d
small doc style fixes
2016-03-05 19:54:06 -03:00
nyov
5876b9aa30
Update documentation links
2016-03-03 16:28:33 +00:00
Mikhail Korobov
7ca9ae1976
DOC typo fix
2016-01-27 17:54:28 +05:00
Mikhail Korobov
4bcbb77bcc
response.text. Fixes GH-1729.
2016-01-27 01:28:11 +05:00
Elias Dorneles
d4c4ca8062
fix version number to appear new feature
2016-01-21 09:42:15 -02:00
Capi Etheriel
659715ecd9
implements FormRequest.from_response CSS support
2016-01-21 01:05:20 -02:00
Marius Gedminas
0620e76433
Fix list formatting
2015-09-29 03:33:30 +05:00
Julia Medina
d3f576a816
Move scrapy/spider.py to scrapy/spiders/__init__.py
2015-05-09 04:20:09 -03:00
Julia Medina
bd0b639b21
Fix logging usage across docs
2015-04-22 17:24:41 -03:00
Julia Medina
cda3922507
Add Response.urljoin() helper
2015-03-19 19:07:52 -03:00
Shadab Zafar
5a58d64131
Fix some redirection links in documentation
...
Fixes #606
2015-03-18 19:41:26 -03:00
nramirezuy
c13e23641b
httpcache dont_cache meta #19 #689
2015-03-16 11:50:04 -03:00
Elias Dorneles
f7031c08ff
updating list of Request.meta special keys
2015-03-10 22:29:07 -03:00
Mikhail Korobov
7d68b084a4
DOC document download_timeout Request.meta key and download_timeout spider attribute.
2014-10-07 04:23:11 +06:00
Mikhail Korobov
36eec8f413
dont_obey_robotstxt meta key; don't process requests to /robots.txt
2014-09-23 00:10:43 +06:00
John-Scott Atlakson
a312ebfb43
Update request-response.rst
...
Fixed minor typo
2014-09-14 22:06:31 +06:00
Mikhail Korobov
774ab74ad2
Merge pull request #864 from younghz/master
...
Duplicate comma in request-response.rst
2014-08-28 18:52:51 +06:00
Uyounghz
d49766a6ac
Duplicate comma in request-response.rst
2014-08-28 19:58:58 +08:00
Rocio Aramberri
51b0bd281d
fix dont settings on meta behaviour, add docs and tests
2014-08-15 13:47:42 -07:00
Rendaw
8bdb6e2e3e
Elaborated request priority value.
2014-05-07 19:14:45 +09:00
Pablo Hoffman
eb07e09166
Merge pull request #663 from pawl/patch-1
...
fixed typo
2014-04-24 17:59:36 -04:00
Daniel Graña
b4593c2ae7
document shortcuts in TextResponse class
2014-04-24 00:15:00 -03:00
Mikhail Korobov
2d3803672b
DOC use top-level shortcuts in docs
2014-04-15 01:09:35 +06:00
Paul Brown
a1ee354609
fixed typo
2014-03-20 15:16:48 -05:00
Julia Medina
ca1c1a82b5
FormRequest doc improvements
...
Clickdata doc enhancements:
* Fix xml attributes mention
* nr is 0-indexed reference
2014-03-12 12:34:50 -03:00
Julia Medina
e29ab4d112
New doc: clickdata in Formrequest.from_response
...
Documentation about:
* clickdata parameter in Formrequest.from_response
* nr attribute in clickdata dict
* default behaviour when clickdata is None
2014-03-12 06:43:50 -03:00
Capi Etheriel
72b6c96d9a
Running lucasdemarchi/codespell to fix typos in docs
2014-03-06 12:40:55 -03:00
Mikhail Korobov
a27d91f0a6
Rename BaseSpider to Spider. See GH-495.
2013-12-30 19:46:41 +06:00
Pablo Hoffman
e42e3743fe
quick documentation for #475
2013-12-24 12:19:15 -02:00
Mikhail Korobov
086b8a20d4
typo fix in TextResponse docs
2013-10-17 04:50:30 +06:00
