1
0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-06 11:00:46 +00:00

10712 Commits

Author SHA1 Message Date
Andrey Rakhmatullin
3d8dbd5648 flake8-bugbear 2024-06-20 00:27:38 +05:00
Andrey Rakhmatullin
1ef9c337ca flake8-docstrings 2024-06-19 23:57:40 +05:00
Andrey Rakhmatullin
1c70d3e605 flake8-comprehensions 2024-06-19 23:36:36 +05:00
Andrey Rakhmatullin
a617e04d2e flake8-string-format 2024-06-19 23:28:58 +05:00
Andrey Rakhmatullin
d132190625 flake8-debugger 2024-06-19 23:26:25 +05:00
Andrey Rakhmatullin
a364560fad
Unpin markupsafe in extra-deps. (#6403) 2024-06-17 11:38:10 +02:00
Andrey Rakhmatullin
365c9e62ad
Removing empty example reference (#6402)
Co-authored-by: Michael Duane Mooring <mikeumus@gmail.com>
2024-06-17 11:37:11 +02:00
Andrey Rakhmatullin
1282ddf8f7
Add parameters to most Deferred in scrapy/core. (#6395) 2024-06-10 10:27:50 +02:00
Andrey Rakhmatullin
ddc98fe91b
Deprecate scrapy.utils.defer.process_chain_both(). (#6397) 2024-06-10 10:16:26 +02:00
Andrey Rakhmatullin
163e7d925e
Merge pull request #6392 from wRAR/pyupgrade
Add pyupgrade.
2024-06-07 11:16:46 +04:00
Andrey Rakhmatullin
5850b8f3e6 Merge branch 'master' into pyupgrade 2024-06-07 11:20:31 +05:00
Andrey Rakhmatullin
ed3a7acaf3 Remove the Python version check from scrapy/__init__.py. 2024-06-07 11:19:37 +05:00
Andrey Rakhmatullin
a4778d2bdf
Merge pull request #6143 from wRAR/typing-request-response-cls
Improve type hints for copy() and replace() in Request and Response.
2024-06-06 22:58:14 +04:00
Andrey Rakhmatullin
1268b23304 Merge branch 'master' into typing-request-response-cls 2024-06-06 21:57:53 +05:00
Andrey Rakhmatullin
b24ecca4d0
Merge pull request #6391 from wRAR/typing-generics-iterables
Add parameters to iterable generics, replace generators with iterables.
2024-06-06 20:54:16 +04:00
Andrey Rakhmatullin
23b1214e90 Add a comment about pyupgrade and scrapy/__init__.py. 2024-06-06 21:44:07 +05:00
Andrey Rakhmatullin
d9d7bd170b
Merge pull request #6394 from Laerte/master
Document missing parts of response.json method
2024-06-06 09:55:56 +04:00
Laerte Pereira
144ff6c756 Document missing parts of response.json method 2024-06-05 21:09:10 -03:00
Andrey Rakhmatullin
feb0b8f7dc Add pyupgrade. 2024-06-05 22:57:18 +05:00
Andrey Rakhmatullin
480a11b68b Add mssing __future__ imports. 2024-06-05 22:48:16 +05:00
Andrey Rakhmatullin
262c10d85b Use typing.Coroutine instead of types.CoroutineType. 2024-06-05 22:11:34 +05:00
Andrey Rakhmatullin
de146ad7ce Bump typing deps. 2024-06-05 22:09:19 +05:00
Andrey Rakhmatullin
2e214210f6 Add parameters to iterable generics, replace generators with iterables. 2024-06-05 21:44:04 +05:00
Suvan Banerjee
3f76853bd2
Handle AttributeError: 'NoneType' in contract parsing (#6388) 2024-06-05 06:34:46 +02:00
Andrey Rakhmatullin
e56b425198
Full typing for scrapy/pipelines. (#6387) 2024-06-05 06:33:45 +02:00
Adrián Chaves
2b9e32f1ca
Merge pull request #6385 from wRAR/typing-generics-collections
Add parameters to various generics.
2024-06-03 13:29:38 +02:00
Andrey Rakhmatullin
492c3bce9d
Don't run callbacks of requests from get_media_requests(). (#6386) 2024-06-03 13:28:20 +02:00
Andrey Rakhmatullin
019f23e3b7 Add parameters to some of typing.Callable. 2024-06-02 18:42:01 +05:00
Andrey Rakhmatullin
859a77ee42 Use a TypedDict for the verbose cookie form. 2024-05-31 22:23:26 +05:00
Andrey Rakhmatullin
751c91e614 Add parameters to misc generics. 2024-05-31 21:57:14 +05:00
Andrey Rakhmatullin
70c56faf48 Add parameters to typing.IO. 2024-05-31 21:41:27 +05:00
Andrey Rakhmatullin
4164e63725 Add parameters to typing.Tuple. 2024-05-31 21:23:55 +05:00
Andrey Rakhmatullin
98c755e5fb Add parameters to typing.List. 2024-05-31 21:20:22 +05:00
Andrey Rakhmatullin
da42e8f124 Add parameters to typing.Dict. 2024-05-31 21:11:50 +05:00
Andrey Rakhmatullin
b950ed77b6
Merge branch 'master' into typing-request-response-cls 2024-05-31 13:07:55 +04:00
Andrey Rakhmatullin
b4293e8f9e
Misc typing improvements. (#6384) 2024-05-31 08:50:36 +02:00
Andrey Rakhmatullin
a011fa6f78
Merge pull request #6371 from wRAR/typing-contracts
Full typing for scrapy/contracts.
2024-05-30 20:59:50 +04:00
Andrey Rakhmatullin
469e8a23f8
Merge pull request #6381 from 0xdeb/referer-policy-tokens
Add support for multiple referer policy tokens.
2024-05-29 17:13:09 +04:00
Fabian Schneebauer
62a028b99d
Add spec link to scrapy/spidermiddlewares/referer.py
Co-authored-by: Adrián Chaves <adrian@chaves.io>
2024-05-29 13:19:27 +02:00
Fabian Schneebauer
0d58af8697 Add support for multiple referer policy tokens. 2024-05-29 10:59:32 +02:00
Andrey Rakhmatullin
1be8aee09c Skip uvloop and bpython on 3.13. 2024-05-28 14:56:23 +05:00
Andrey Rakhmatullin
5755e224d5 Help with building lxml on 3.13beta1 for checks too. 2024-05-28 14:50:09 +05:00
Andrey Rakhmatullin
e6e9fd75db Skip mitmproxy and Pillow on 3.13. 2024-05-28 14:37:05 +05:00
Andrey Rakhmatullin
42347de53f Install pre-release cffi on 3.13. 2024-05-28 14:37:05 +05:00
Andrey Rakhmatullin
d9b5538e3c Bump twinecheck deps. 2024-05-28 14:04:58 +05:00
Sanchay Kumar
cadb0dd707
Fix overridable methods in MediaPipeline (#6368) 2024-05-28 10:42:58 +02:00
Andrey Rakhmatullin
986d1ee1dd
Move CI from the decommissioned macos-11 to macos-latest. (#6372) 2024-05-28 10:37:19 +02:00
Andrey Rakhmatullin
9ba4dd311d
Install typing stubs for boto3 and botocore. (#6370) 2024-05-28 10:27:49 +02:00
Andrey Rakhmatullin
f9a9860306
Merge pull request #6374 from Laerte/master
docs: Remove top-level reactor imports from CrawlerProces/CrawlerRunner examples
2024-05-27 14:36:34 +04:00
Laerte Pereira
6cd0857850 Move path 2024-05-26 19:57:16 -03:00