scrapy

CrawlerProcess

Small 3.7 and 3.8 cleanup.

2024-10-17 23:22:37 +05:00

CrawlerRunner

Replace isort with the ruff isort rules.

2024-12-29 14:15:16 +05:00

keys

Commit mitmproxy-dhparam.pem.

2024-11-15 21:08:18 +05:00

mocks

Drop Python 3.8 Support (#6472 )

2024-10-16 10:03:16 +02:00

sample_data

Add support for multiple-compressed responses (#6063 )

2024-02-22 12:16:24 +01:00

test_cmdline

Enable RUF Ruff rules.

2025-01-01 21:41:08 +05:00

test_cmdline_crawl_with_pipeline

Add pylint rules to ruff, refresh the ignore list of pylint itself (#6584 )

2024-12-16 09:44:36 +01:00

test_settings

Merge pull request #6601 from wRAR/ruff-rules-5

2025-01-02 17:38:15 +04:00

test_spiderloader

Allow updating pre-crawler settings from add-ons (#6568 )

2025-01-22 12:07:44 +01:00

test_utils_misc

Drop the remaining unittest.main() blocks. (#6602 )

2025-01-02 12:45:04 +01:00

__init__.py

Fix tracking of coverage in subprocesses.

2025-01-07 20:41:48 +05:00

ftpserver.py

adding black formatter to all the code

2022-11-29 11:30:46 -03:00

ignores.txt

Remove deprecated classes BaseSgmlLinkExtractor, RegexLinkExtractor and SgmlLinkExtractor (#4356 )

2020-07-17 12:40:04 +02:00

mockserver.py

Drop Python 3.8 Support (#6472 )

2024-10-16 10:03:16 +02:00

pipelines.py

reaplying black, fixing conflicts and ignoring bandit checks on test directory

2023-01-20 10:55:46 -03:00

spiders.py

Bump ruff, switch from black to ruff-format (#6631 )

2025-01-27 11:07:09 +01:00

test_addons.py

Enable SIM Ruff rules.

2025-01-02 01:03:36 +05:00

test_closespider.py

Make mockserver instances per-class.

2025-02-02 14:10:09 +05:00

test_command_check.py

Run black.

2024-03-14 17:44:57 +05:00

test_command_fetch.py

Upgrade CI tools

2023-02-02 06:37:40 +01:00

test_command_parse.py

scrapy parse: fix the signature of callbacks from the CLI (#6182 )

2024-01-15 13:37:03 +01:00

test_command_shell.py

Merge pull request #6112 from wRAR/test-shutdown-forced

2023-11-16 19:16:52 +04:00

test_command_version.py

flake8-bugbear

2024-06-20 00:27:38 +05:00

test_commands.py

Fix error when running scrapy bench (#6633 )

2025-01-27 11:21:30 +01:00

test_contracts.py

Bump ruff, switch from black to ruff-format (#6631 )

2025-01-27 11:07:09 +01:00

test_core_downloader.py

Silence the readBody warning.

2025-01-28 02:08:49 +05:00

test_crawl.py

Make mockserver instances per-class.

2025-02-02 14:10:09 +05:00

test_crawler.py

Enable PTH Ruff rules.

2025-01-01 22:21:51 +05:00

test_dependencies.py

Remove obsolete top-level run code in tests.

2023-01-25 22:14:24 +04:00

test_downloader_handlers_http2.py

Refactor downloader tests (#6647 )

2025-02-03 20:11:47 +05:00

test_downloader_handlers.py

Refactor downloader tests (#6647 )

2025-02-03 20:11:47 +05:00

test_downloadermiddleware_ajaxcrawlable.py

Deprecate AjaxCrawlMiddleware.

2025-02-03 14:08:05 +05:00

test_downloadermiddleware_cookies.py

Address some previously ignored pylint messages. (#6531 )

2024-11-11 11:49:52 +01:00

test_downloadermiddleware_defaultheaders.py

sort imports with isort

2023-01-25 14:25:15 -06:00

test_downloadermiddleware_downloadtimeout.py

sort imports with isort

2023-01-25 14:25:15 -06:00

test_downloadermiddleware_httpauth.py

Fix tests

2024-02-28 16:36:19 -03:00

test_downloadermiddleware_httpcache.py

Remove a duplicate test.

2025-02-02 18:48:26 +05:00

test_downloadermiddleware_httpcompression.py

Add flake8-raise rules to ruff.

2024-12-12 20:33:35 +05:00

test_downloadermiddleware_httpproxy.py

Bump ruff, switch from black to ruff-format (#6631 )

2025-01-27 11:07:09 +01:00

test_downloadermiddleware_offsite.py

new allow_offsite parameter in OffsiteMiddleware (#6151 )

2025-01-08 21:28:51 +05:00

test_downloadermiddleware_redirect.py

Bump ruff, switch from black to ruff-format (#6631 )

2025-01-27 11:07:09 +01:00

test_downloadermiddleware_retry.py

Merge pull request #6601 from wRAR/ruff-rules-5

2025-01-02 17:38:15 +04:00

test_downloadermiddleware_robotstxt.py

Add flake8-return rules to ruff.

2024-12-12 20:32:25 +05:00

test_downloadermiddleware_stats.py

Removed some deprecated functions and functionalities (#6116 )

2023-10-30 09:59:11 +01:00

test_downloadermiddleware_useragent.py

sort imports with isort

2023-01-25 14:25:15 -06:00

test_downloadermiddleware.py

Add pylint rules to ruff, refresh the ignore list of pylint itself (#6584 )

2024-12-16 09:44:36 +01:00

test_downloaderslotssettings.py

Make mockserver instances per-class.

2025-02-02 14:10:09 +05:00

test_dupefilters.py

Deprecate BaseDupeFilter.log() and improve dupefilter docs (#4151 )

2025-01-14 19:36:56 +05:00

test_engine_stop_download_bytes.py

Refactor EngineTest tests.

2025-02-01 16:07:55 +05:00

test_engine_stop_download_headers.py

Refactor EngineTest tests.

2025-02-01 16:07:55 +05:00

test_engine.py

Refactor EngineTest tests.

2025-02-01 16:07:55 +05:00

test_exporters.py

Bump ruff, switch from black to ruff-format (#6631 )

2025-01-27 11:07:09 +01:00

test_extension_periodic_log.py

Add pylint rules to ruff, refresh the ignore list of pylint itself (#6584 )

2024-12-16 09:44:36 +01:00

test_extension_telnet.py

Add flake8-pie rules to ruff.

2024-12-12 20:33:35 +05:00

test_extension_throttle.py

dont_throttle → autothrottle_dont_adjust_delay

2024-11-07 16:17:16 +01:00

test_feedexport.py

Make mockserver instances per-class.

2025-02-02 14:10:09 +05:00

test_http2_client_protocol.py

Drop Python 3.8 Support (#6472 )

2024-10-16 10:03:16 +02:00

test_http_cookies.py

Fix WrappedRequest.get_header raising TypeError if default is None (#6310 )

2024-04-08 11:47:53 +02:00

test_http_headers.py

sort imports with isort

2023-01-25 14:25:15 -06:00

test_http_request.py

Deprecate and disable escape_ajax().

2025-02-03 14:08:05 +05:00

test_http_response.py

Bump ruff, switch from black to ruff-format (#6631 )

2025-01-27 11:07:09 +01:00

test_item.py

Bump ruff, switch from black to ruff-format (#6631 )

2025-01-27 11:07:09 +01:00

test_link.py

Split ruff and pylint ignores into two categories, some pylint cleanup.

2025-01-02 01:03:42 +05:00

test_linkextractors.py

Address some previously ignored pylint messages. (#6531 )

2024-11-11 11:49:52 +01:00

test_loader_deprecated.py

Drop the remaining unittest.main() blocks. (#6602 )

2025-01-02 12:45:04 +01:00

test_loader.py

Drop the remaining unittest.main() blocks. (#6602 )

2025-01-02 12:45:04 +01:00

test_logformatter.py

Make mockserver instances per-class.

2025-02-02 14:10:09 +05:00

test_logstats.py

Fix the calculate_final_stats method (#6599 )

2025-01-02 12:36:28 +01:00

test_mail.py

Drop the remaining unittest.main() blocks. (#6602 )

2025-01-02 12:45:04 +01:00

test_middleware.py

Remove build_from_settings().

2024-11-11 16:44:47 +05:00

test_pipeline_crawl.py

Make mockserver instances per-class.

2025-02-02 14:10:09 +05:00

test_pipeline_files.py

Bump ruff, switch from black to ruff-format (#6631 )

2025-01-27 11:07:09 +01:00

test_pipeline_images.py

Add flake8-pie rules to ruff.

2024-12-12 20:33:35 +05:00

test_pipeline_media.py

Bump ruff, switch from black to ruff-format (#6631 )

2025-01-27 11:07:09 +01:00

test_pipelines.py

Make mockserver instances per-class.

2025-02-02 14:10:09 +05:00

test_pqueues.py

sort imports with isort

2023-01-25 14:25:15 -06:00

test_proxy_connect.py

Make mockserver instances per-class.

2025-02-02 14:10:09 +05:00

test_request_attribute_binding.py

Make mockserver instances per-class.

2025-02-02 14:10:09 +05:00

test_request_cb_kwargs.py

Make mockserver instances per-class.

2025-02-02 14:10:09 +05:00

test_request_dict.py

Address some previously ignored pylint messages. (#6531 )

2024-11-11 11:49:52 +01:00

test_request_left.py

Make mockserver instances per-class.

2025-02-02 14:10:09 +05:00

test_responsetypes.py

Drop the remaining unittest.main() blocks. (#6602 )

2025-01-02 12:45:04 +01:00

test_robotstxt_interface.py

Bump ruff, switch from black to ruff-format (#6631 )

2025-01-27 11:07:09 +01:00

test_scheduler_base.py

Remove --keep-runtime-typing from pyupgrade.

2024-10-17 21:26:02 +05:00

test_scheduler.py

Remove useless pylint: disable lines.

2025-01-02 12:32:25 +05:00

test_scrapy__getattr__.py

Split ruff and pylint ignores into two categories, some pylint cleanup.

2025-01-02 01:03:42 +05:00

test_selector.py

Bump ruff, switch from black to ruff-format (#6631 )

2025-01-27 11:07:09 +01:00

test_signals.py

Make mockserver instances per-class.

2025-02-02 14:10:09 +05:00

test_spider.py

flake8-comprehensions

2024-06-19 23:36:36 +05:00

test_spidermiddleware_depth.py

sort imports with isort

2023-01-25 14:25:15 -06:00

test_spidermiddleware_httperror.py

Make mockserver instances per-class.

2025-02-02 14:10:09 +05:00

test_spidermiddleware_offsite.py

new allow_offsite parameter in OffsiteMiddleware (#6151 )

2025-01-08 21:28:51 +05:00

test_spidermiddleware_output_chain.py

Add pylint rules to ruff, refresh the ignore list of pylint itself (#6584 )

2024-12-16 09:44:36 +01:00

test_spidermiddleware_referer.py

Reformat long REFERRER_POLICY.

2025-01-02 18:14:18 +05:00

test_spidermiddleware_urllength.py

Add from_crawler() to components that only had from_settings().

2024-11-12 21:12:32 +05:00

test_spidermiddleware.py

Add pylint rules to ruff, refresh the ignore list of pylint itself (#6584 )

2024-12-16 09:44:36 +01:00

test_spiderstate.py

Remove usage of deprecated mktemp (#5285 )

2024-02-26 19:53:06 +01:00

test_squeues_request.py

Split ruff and pylint ignores into two categories, some pylint cleanup.

2025-01-02 01:03:42 +05:00

test_squeues.py

Upgrade CI tools

2023-02-02 06:37:40 +01:00

test_stats.py

Remove datetime.utcnow() usage (#6014 )

2023-08-21 15:51:49 +02:00

test_toplevel.py

sort imports with isort

2023-01-25 14:25:15 -06:00

test_urlparse_monkeypatches.py

sort imports with isort

2023-01-25 14:25:15 -06:00

test_utils_asyncgen.py

Add Perflint rules to ruff.

2024-12-12 21:03:34 +05:00

test_utils_asyncio.py

Add pylint rules to ruff, refresh the ignore list of pylint itself (#6584 )

2024-12-16 09:44:36 +01:00

test_utils_conf.py

Drop the remaining unittest.main() blocks. (#6602 )

2025-01-02 12:45:04 +01:00

test_utils_console.py

Drop the remaining unittest.main() blocks. (#6602 )

2025-01-02 12:45:04 +01:00

test_utils_curl.py

feat: added test_post_data_raw_with_string_prefix

2023-04-18 22:28:16 -05:00

test_utils_datatypes.py

Drop the remaining unittest.main() blocks. (#6602 )

2025-01-02 12:45:04 +01:00

test_utils_defer.py

Add pylint rules to ruff, refresh the ignore list of pylint itself (#6584 )

2024-12-16 09:44:36 +01:00

test_utils_deprecate.py

Enable SIM Ruff rules.

2025-01-02 01:03:36 +05:00

test_utils_display.py

sort imports with isort

2023-01-25 14:25:15 -06:00

test_utils_gz.py

IOError and other cleanup (#4716 )

2023-06-21 20:08:53 +02:00

test_utils_httpobj.py

Drop the remaining unittest.main() blocks. (#6602 )

2025-01-02 12:45:04 +01:00

test_utils_iterators.py

Enable SIM Ruff rules.

2025-01-02 01:03:36 +05:00

test_utils_log.py

Drop Python 3.8 Support (#6472 )

2024-10-16 10:03:16 +02:00

test_utils_project.py

Enable PTH Ruff rules.

2025-01-01 22:21:51 +05:00

test_utils_python.py

Add Perflint rules to ruff.

2024-12-12 21:03:34 +05:00

test_utils_request.py

Drop the remaining unittest.main() blocks. (#6602 )

2025-01-02 12:45:04 +01:00

test_utils_response.py

Bump ruff, switch from black to ruff-format (#6631 )

2025-01-27 11:07:09 +01:00

test_utils_serialize.py

Fix and remove most of the entries from the mypy ignore list (#6137 )

2023-11-07 09:34:35 +01:00

test_utils_signal.py

Bump pylint, cleanup the ignored tags.

2024-02-28 14:31:21 +05:00

test_utils_sitemap.py

Drop the remaining unittest.main() blocks. (#6602 )

2025-01-02 12:45:04 +01:00

test_utils_spider.py

Drop the remaining unittest.main() blocks. (#6602 )

2025-01-02 12:45:04 +01:00

test_utils_template.py

Drop the remaining unittest.main() blocks. (#6602 )

2025-01-02 12:45:04 +01:00

test_utils_trackref.py

Enable RUF Ruff rules.

2025-01-01 21:41:08 +05:00

test_utils_url.py

Bump ruff, switch from black to ruff-format (#6631 )

2025-01-27 11:07:09 +01:00

test_webclient.py

Remove scrapy.core.downloader.webclient._parse().

2025-01-28 22:22:09 +05:00

upper-constraints.txt

upper-constraints.txt: restrict botocore further

2021-03-18 23:42:29 +01:00