1
0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-06 11:00:46 +00:00

dont_throttle → autothrottle_dont_adjust_delay

This commit is contained in:
Adrián Chaves 2024-11-07 16:17:16 +01:00
parent b244ea7ac0
commit 2a4b7fe0f8
4 changed files with 11 additions and 7 deletions

View File

@ -47,19 +47,19 @@ effect, but there are some important differences:
AutoThrottle doesn't have these issues.
.. reqmeta:: dont_throttle
.. reqmeta:: autothrottle_dont_adjust_delay
Disabling the throttling of a request
=====================================
To disable AutoThrottle for a specific request, set the ``dont_throttle``
request metadata key to ``True``:
To disable AutoThrottle for a specific request, set the
``autothrottle_dont_adjust_delay`` request metadata key to ``True``:
.. code-block:: python
from scrapy import Request
Request("https://example.com", meta={"dont_throttle": True})
Request("https://example.com", meta={"autothrottle_dont_adjust_delay": True})
Note, however, that AutoThrottle still determines the starting delay of every
slot by setting the ``download_delay`` attribute on the running spider. You

View File

@ -668,6 +668,7 @@ are some special keys recognized by Scrapy and its built-in extensions.
Those are:
* :reqmeta:`autothrottle_dont_adjust_delay`
* :reqmeta:`bindaddress`
* :reqmeta:`cookiejar`
* :reqmeta:`dont_cache`
@ -675,7 +676,6 @@ Those are:
* :reqmeta:`dont_obey_robotstxt`
* :reqmeta:`dont_redirect`
* :reqmeta:`dont_retry`
* :reqmeta:`dont_throttle`
* :reqmeta:`download_fail_on_dataloss`
* :reqmeta:`download_latency`
* :reqmeta:`download_maxsize`

View File

@ -67,7 +67,7 @@ class AutoThrottle:
if (
latency is None
or slot is None
or request.meta.get("dont_throttle", False) is True
or request.meta.get("autothrottle_dont_adjust_delay", False) is True
):
return

View File

@ -165,7 +165,11 @@ def test_startdelay_definition(min_spider, min_setting, start_setting, expected)
({"download_slot": "foo"}, "foo"),
({"download_latency": 1.0, "download_slot": "foo"}, None),
(
{"download_latency": 1.0, "download_slot": "foo", "dont_throttle": True},
{
"download_latency": 1.0,
"download_slot": "foo",
"autothrottle_dont_adjust_delay": True,
},
"foo",
),
),