Paul Tremberth
|
c68f99eed8
|
Refactor settings tests
|
2017-03-03 17:03:25 +01:00 |
|
Paul Tremberth
|
c3b6feca0e
|
Fix setting lookup for MEDIA_ALLOWED_REDIRECTS
|
2017-03-03 16:29:07 +01:00 |
|
Paul Tremberth
|
f7e11b198e
|
Cleanup
|
2017-03-03 16:00:59 +01:00 |
|
Paul Tremberth
|
ecde166ee1
|
Refactor without MEDIA_HTTPSTATUS_LIST setting
|
2017-03-03 15:52:05 +01:00 |
|
Paul Tremberth
|
72fbb687d7
|
Revert "expose allowed_status tuple for media pipeline"
This reverts commit 052809c73ed20b9a728a8fd7df3de5f45f2dad8d.
|
2017-03-03 15:52:05 +01:00 |
|
Paul Tremberth
|
c64ebee062
|
Refactor (WIP)
|
2017-03-03 15:52:05 +01:00 |
|
Bernardas
|
11b31c9fbd
|
fix redirect change
|
2017-03-03 15:52:05 +01:00 |
|
Bernardas
|
3cef1cd451
|
adjust variable wording and redirect logic
|
2017-03-03 15:52:05 +01:00 |
|
Bernardas
|
f0b4077f81
|
expose allowed_status tuple for media pipeline
|
2017-03-03 15:52:05 +01:00 |
|
Bernardas
|
2e052c8615
|
fix error when settings are not provided
|
2017-03-03 15:52:05 +01:00 |
|
Bernardas
|
8542780854
|
typo and clarify handling
|
2017-03-03 15:52:05 +01:00 |
|
Bernardas
|
6a42214716
|
add tests for media pipeline MEDIA_ALLOW_REDIRECTS and MEDIA_HTTPSTATUS_LIST settings
|
2017-03-03 15:52:05 +01:00 |
|
Bernardas
|
25ed491219
|
add description for media pipeline MEDIA_ALLOW_REDIRECTS and MEDIA_HTTPSTATUS_LIST settings
|
2017-03-03 15:52:05 +01:00 |
|
Bernardas
|
5d0058492c
|
add media pipeline settings to enable redirection and handling of certain http statuses
|
2017-03-03 15:52:05 +01:00 |
|
Mikhail Korobov
|
7e8453cf1e
|
Merge pull request #2306 from redapple/referrer-policy
[MRG] Referrer policies in RefererMiddleware
|
2017-03-03 04:05:13 +05:00 |
|
Paul Tremberth
|
fad499ab60
|
Rename arguments (bis)
|
2017-03-02 23:06:04 +01:00 |
|
Paul Tremberth
|
db176f872b
|
Remove Legacy Policy which is equivalent to UnsafeUrl Policy
|
2017-03-02 22:56:23 +01:00 |
|
Mikhail Korobov
|
93024c242b
|
Merge pull request #2537 from scrapy/no-canonicalize
[MRG+1] Set canonicalize=False for LinkExtractor
|
2017-03-03 02:53:36 +05:00 |
|
Paul Tremberth
|
c2c503192f
|
Rename arguments
|
2017-03-02 22:53:27 +01:00 |
|
Paul Tremberth
|
2d55d838ca
|
Fix strip_url() tests
|
2017-03-01 20:59:52 +01:00 |
|
Paul Tremberth
|
efa50039ec
|
Add tests for policy fallback on unknown policies from meta and headers
|
2017-03-01 17:51:23 +01:00 |
|
Paul Tremberth
|
6916dd6240
|
Warn or fail with exception on unknown policies
|
2017-03-01 17:51:23 +01:00 |
|
Paul Tremberth
|
8226e77010
|
Add test for Referer header on HTTP redirections
|
2017-03-01 17:51:23 +01:00 |
|
Paul Tremberth
|
d2aa51c0fb
|
Update tests
|
2017-03-01 17:51:23 +01:00 |
|
Paul Tremberth
|
04e4d08612
|
Pass URLs around instead of Request/Responses
|
2017-03-01 17:51:23 +01:00 |
|
Paul Tremberth
|
bc200d1155
|
Rename setting to REFERRER_POLICY (with 2 Rs)
|
2017-03-01 17:51:23 +01:00 |
|
Paul Tremberth
|
537683f945
|
Add autoclass directives to document built-in policies
|
2017-03-01 17:51:23 +01:00 |
|
Paul Tremberth
|
3dc09eeceb
|
Use table for referrer policy options
|
2017-03-01 17:51:23 +01:00 |
|
Paul Tremberth
|
605935f015
|
Edit text
|
2017-03-01 17:51:23 +01:00 |
|
Paul Tremberth
|
eb07285a63
|
Reword warning on no-referrer-when-downgrade policy
|
2017-03-01 17:51:23 +01:00 |
|
Paul Tremberth
|
03ff19d188
|
Update docs for new "referrer_policy" Request.meta key
|
2017-03-01 17:51:23 +01:00 |
|
Paul Tremberth
|
e249abc32b
|
Update docs
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
c86f568b9c
|
Update docs with "strict-..." policies
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
b6c761d2b4
|
Fix tests
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
ebcacd3f54
|
Update StrictOriginPolicy
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
deb8567116
|
Update NoReferrerWhenDowngradePolicy
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
77aec5a796
|
Fix implementation
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
5cef67ae75
|
Update Referrer tests for "strict-" policies
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
c808a97c74
|
Add new "strict-" policies
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
0a0b60a59f
|
Add tests for stripping userinfo with percent-encoded delimiters
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
8864d0e8c1
|
Rename helper function to strip_url() + add more tests
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
5dd7311cd4
|
Move URL credentials stripping to a helper function
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
c9c59db489
|
Update documentation about REFERER_POLICY setting
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
285d5bc03a
|
Patch "Referer" header on HTTP redirects if necessary
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
d3d4d66ce8
|
Add tests for referrer-policy set in response HTTP headers
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
e50e670eff
|
Add test for custom referrer policy via settings
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
ec8b4c1a9b
|
Change __init__ default "settings" arg handling
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
0344f57fef
|
Support case-insensitive policy names in settings
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
e72b6e3361
|
Add tests for referrer policy via settings and via Request meta
|
2017-03-01 17:50:39 +01:00 |
|
Paul Tremberth
|
842ce131aa
|
Make default referrer policy customizable via settings
|
2017-03-01 17:50:39 +01:00 |
|