Andrey Rakhmatullin
ab5cb7c7d9
Merge pull request #6540 from wRAR/build_from_settings
...
Remove build_from_settings() and deprecate from_settings() code
2024-11-14 20:40:54 +05:00
Andrey Rakhmatullin
929d665a74
Address PR feedback.
2024-11-14 19:35:56 +05:00
Andrey Rakhmatullin
2ad5f0c12b
Extract duplicated code.
2024-11-14 13:03:04 +05:00
Andrey Rakhmatullin
6aa4d2b4ab
Prefer crawler.settings over settings in media pipelines.
2024-11-14 02:03:43 +05:00
Andrey Rakhmatullin
28fafbb8c5
Modernize the media pipeline initialization API.
2024-11-14 01:29:51 +05:00
Robert Palmer
261c4b61dc
Enhancement: Update docs to include IgnoreRequest details ( #6506 )
...
Co-authored-by: Adrián Chaves <adrian@chaves.io>
2024-11-13 22:47:39 +05:00
Andrey Rakhmatullin
8700a5b7a9
Deprecate build_from_crawler() calling from_settings().
2024-11-13 00:20:10 +05:00
Andrey Rakhmatullin
eda1a8a7c5
Deprecate MiddlewareManager.from_settings().
2024-11-12 22:57:39 +05:00
Andrey Rakhmatullin
499e7e8aa6
Add from_crawler() to components that only had from_settings().
2024-11-12 21:12:32 +05:00
Andrey Rakhmatullin
f796d8780c
Add tests for MediaPipeline.from_crawler() and related code.
2024-11-12 21:08:04 +05:00
Andrey Rakhmatullin
83d4939d41
Deprecate scrapy.extensions.feedexport.build_storage() and simplify _get_storage().
2024-11-11 22:15:08 +05:00
Andrey Rakhmatullin
eda3a89b3f
Remove build_from_settings().
2024-11-11 16:44:47 +05:00
Andrey Rakhmatullin
b042ad255d
Address some previously ignored pylint messages. ( #6531 )
2024-11-11 11:49:52 +01:00
Mikhail Korobov
bcef96570b
Merge pull request #6527 from Gallaecio/dont-throttle-meta
...
Replace Slot.throttle with Request.meta['dont_throttle']
2024-11-08 14:25:37 +05:00
Adrián Chaves
dc3ebb6cf7
Refactor the docs
2024-11-07 16:38:48 +01:00
Adrián Chaves
2a4b7fe0f8
dont_throttle → autothrottle_dont_adjust_delay
2024-11-07 16:17:16 +01:00
Adrián Chaves
b244ea7ac0
Add the missing import to the docs example
2024-11-05 20:05:58 +01:00
Adrián Chaves
5862216bb1
Fix docs example
2024-11-05 19:55:28 +01:00
Adrián Chaves
f57fc454be
Replace Slot.throttle with Request.meta['dont_throttle']
2024-11-05 19:44:30 +01:00
Rohit Kumar Singh
d2156696c4
Deprecate unused scrapy utils ( #6519 )
...
* Added deprecation warnings for unused Scrapy.utils
* Grammatical corrections
* Exceptions class connected
* Deprecation of ScrapyJSONDecoder
* request_authenticate function deprecation
* Making all warning similar
* Added ignore statements for deprecation warning in tests
* Missing stacklevel attr. added
* Added Deprecation message
2024-11-04 21:09:45 +05:00
Andrey Rakhmatullin
e7f5ae0b34
Update the outdated Item docstring. ( #6427 )
...
* Update the outdated Item doscstring.
* Fix the reference links in items.html.
2024-11-04 16:17:56 +05:00
Andrey Rakhmatullin
ce5a132f12
Run and fix linkcheck. ( #6524 )
2024-11-04 11:40:07 +01:00
Rohit Kumar Singh
7701e590fb
Documentation added for Spider State in extensions.rst ( #6522 )
...
* Documentation added for Spider State in extensions.rst
* Made correction in documentation for Spiderstate
* Added appropriate intro for Spider state extension
* Added reference for spiderstate extension
* Added Spiderstate extension hyperlink refrence in jobs.rst
2024-11-02 10:45:27 +05:00
Andrey Rakhmatullin
d85c39f5bc
Deprecation removals. ( #6500 )
...
* Deprecation removals.
* Clean up the default pytest filterwarnings.
* Remove test_get_images_old().
* Redo boto-requiring test filtering.
* Remove an unused function.
* Improve the Crawler.crawl() error message.
* Fix the test.
2024-10-31 18:06:22 +05:00
Laerte Pereira
d2bdbad8c8
Deprecate scrapy.twisted_version ( #6512 )
...
* Deprecate scrapy.twisted_version
* fix: typing
* remove typing
* raise default exception if attribute is not found
* remove redudant ()
* add tests
* rollback exception raised
* add filterwarnings again
* change order
* lint
2024-10-30 00:28:35 +05:00
Rohitkr117
12b087b0f2
Added ignore statements for Windows specific typing issues ( #6516 )
2024-10-29 19:30:32 +01:00
Rohitkr117
65ecd5d528
Fixes for Twisted Version Check and Typing Issues ( #6511 )
2024-10-29 19:08:38 +01:00
ThunderMind
5bbf8124ac
Updated deprecated ast.NameConstant with ast.Constant #6305 ( #6463 )
2024-10-29 10:28:00 +01:00
Andrey Rakhmatullin
fcb5ab6cff
Remove code for unsupported Twisted. ( #6510 )
2024-10-29 10:21:07 +01:00
Andrey Rakhmatullin
0523e1616d
Explictly set html_baseurl on RTD. ( #6507 )
2024-10-29 10:16:03 +01:00
Andrey Rakhmatullin
b4bad97eae
Merge pull request #6166 from Gallaecio/py313
...
Add Python 3.13 to CI
2024-10-22 20:05:56 +05:00
Andrey Rakhmatullin
d10c58ff38
Bump pyftpdlib to the version supporting Python 3.13 on Windows.
2024-10-22 19:07:21 +05:00
Andrey Rakhmatullin
fffacb9dac
Merge remote-tracking branch 'origin/master' into py313
2024-10-22 19:07:10 +05:00
Andrey Rakhmatullin
04d0411bf7
Filter test-time warnings. ( #6501 )
2024-10-21 12:30:49 +02:00
Andrey Rakhmatullin
6d65708cb7
Merge pull request #6499 from wRAR/3.8-cleanup
...
Additional Python 3.7 and 3.8 cleanup, including Reppy support.
2024-10-18 00:08:11 +05:00
Andrey Rakhmatullin
677e977207
Remove dead links to the Reppy doc from the release notes.
2024-10-18 00:03:32 +05:00
Andrey Rakhmatullin
5759b3f0f2
Drop Reppy.
2024-10-17 23:41:23 +05:00
Andrey Rakhmatullin
7e07d48cc5
Small 3.7 and 3.8 cleanup.
2024-10-17 23:22:37 +05:00
Andrey Rakhmatullin
1138a5cf99
Merge pull request #6498 from wRAR/remove-runtime-typing
...
Remove --keep-runtime-typing from pyupgrade.
2024-10-17 23:18:22 +05:00
Andrey Rakhmatullin
7196a11f53
Reorder unions with None.
2024-10-17 21:52:03 +05:00
Andrey Rakhmatullin
c9095ef927
Remove --keep-runtime-typing from pyupgrade.
2024-10-17 21:26:02 +05:00
Julian Ste
c8e87ab21a
Fixed typos ( #6497 )
2024-10-17 17:03:16 +02:00
Andrey Rakhmatullin
f65e64a724
Misc typing improvements. ( #6494 )
2024-10-16 18:38:43 +02:00
Andrey Rakhmatullin
9bd5e5bcdb
Revert uvloop restrictions.
2024-10-16 14:50:57 +05:00
Andrey Rakhmatullin
845b1ffd44
Merge remote-tracking branch 'origin/master' into py313
2024-10-16 14:50:12 +05:00
Vsevolod Breus
5391663072
Drop Python 3.8 Support ( #6472 )
2024-10-16 10:03:16 +02:00
Andrey Rakhmatullin
9736e49b52
Merge pull request #6489 from rettinghaus/pypa_action
...
update gh-action-pypi-publish
2024-10-11 00:24:31 +05:00
Klaus Rettinghaus
5ef5474172
update gh-action-pypi-publish
2024-10-09 20:38:50 +02:00
Andrey Rakhmatullin
7ec6b7e65b
Merge pull request #6487 from wRAR/drop-setuptools-dep
...
Remove the installation dependency on setuptools.
2024-10-08 16:34:11 +05:00
Andrey Rakhmatullin
87651fdf47
Don't use types-setuptools.
2024-10-08 16:04:26 +05:00
