diff --git a/docs/topics/request-response.rst b/docs/topics/request-response.rst index 3d110b02d..67f8ec285 100644 --- a/docs/topics/request-response.rst +++ b/docs/topics/request-response.rst @@ -307,6 +307,7 @@ Those are: * :reqmeta:`proxy` * ``ftp_user`` (See :setting:`FTP_USER` for more info) * ``ftp_password`` (See :setting:`FTP_PASSWORD` for more info) +* :reqmeta:`referrer_policy` .. reqmeta:: bindaddress diff --git a/docs/topics/spider-middleware.rst b/docs/topics/spider-middleware.rst index 0ddf027ea..a4ac45b41 100644 --- a/docs/topics/spider-middleware.rst +++ b/docs/topics/spider-middleware.rst @@ -337,6 +337,8 @@ REFERER_POLICY Default: ``'scrapy.spidermiddlewares.referer.DefaultReferrerPolicy'`` +.. reqmeta:: referrer_policy + `Referrer Policy`_ to apply when populating Request "Referer" header. This setting accepts: @@ -370,6 +372,11 @@ using ``file://`` or ``s3://`` scheme. ``same-origin`` may be a better choice if you want to remove referrer information for cross-domain requests. +.. note:: + You can also override the Referrer Policy per request, + using the special ``"referrer_policy"`` :ref:`Request.meta ` key, + with the same acceptable values as for the ``REFERER_POLICY`` setting. + .. _Referrer Policy: https://www.w3.org/TR/referrer-policy .. _"no-referrer": https://www.w3.org/TR/referrer-policy/#referrer-policy-no-referrer .. _"no-referrer-when-downgrade": https://www.w3.org/TR/referrer-policy/#referrer-policy-no-referrer-when-downgrade