Andrey Rahmatullin
af55d23167
Update the OpenSSL cipher list format link
...
OpenSSL `ciphers(1)` is now almost empty: https://www.openssl.org/docs/manmaster/man1/ciphers.html
Alternative would be linking to 1.1.1 docs specifically.
2020-07-01 19:46:54 +05:00
Eugenio Lacuesta
ec98dabfab
Support for dataclass and attrs items ( #3881 )
2020-06-14 11:45:27 +02:00
Mikhail Korobov
fe6154e4fa
clarify DOWNLOADER_HTTPCLIENTFACTORY docs
2020-05-04 18:18:38 +05:00
Mikhail Korobov
83d7360bb7
Don't mention unsupported package versions in docs
2020-05-04 02:00:11 +05:00
Mikhail Korobov
36abe9235c
Merge pull request #4445 from adityaa30/docs-redirect-links
...
update redirect links to python3
2020-04-15 23:13:55 +05:00
Eugenio Lacuesta
34e81d0d74
Docs: remove duplicated setting definitions
2020-04-10 17:29:02 -03:00
Aditya
80c69d68ad
[docs] refactor python docs links using intersphinx
2020-03-24 18:35:42 +05:30
Aditya
ca08e04198
[docs] update redirect links python2 -> python3
2020-03-20 02:31:35 +05:30
elacuesta
dfbe1d9507
Remove object base class ( #4430 )
2020-03-16 20:12:46 +01:00
Aditya Kumar
f9bf4b8d4d
Remove all top-level imports for twisted.internet.reactor ( #4406 )
2020-03-14 10:39:00 +01:00
Adrián Chaves
6aa0ba4553
Write release notes for Scrapy 2.0.0 ( #4329 )
2020-03-03 09:11:11 +01:00
Akshay Sharma
182445f9d9
Fix a spelling error: ie. → i.e. ( #4338 )
2020-02-18 17:58:31 +01:00
elacuesta
35dafef7f1
Specify Twisted reactor (TWISTED_REACTOR setting) ( #4294 )
...
* Add the ability to install a specific reactor
* Add docs for the TWISTED_REACTOR setting
* Add tests for the TWISTED_REACTOR setting
* Update asyncio reactor test
* Ignore W503 globally
W503 is not PEP8-compliant:
c59c4376ad
2020-02-06 22:42:34 +05:00
Mikhail Korobov
8b8df31961
Merge pull request #4227 from elacuesta/name-resolver
...
Name resolver with IPv6 support
2020-01-25 05:03:13 +05:00
JP Balarini
0f2d871d88
Use PEP 515 style for SCRAPER_SLOT_MAX_ACTIVE_SIZE documentation
2020-01-20 11:28:28 -03:00
Juan Pablo Balarini
eaa8ed02d0
Add ability to change max_active_size by settings
2020-01-20 11:27:58 -03:00
Eugenio Lacuesta
302d3f552b
[doc] DNS_RESOLVER setting
2020-01-18 01:46:24 -03:00
Andrey Rakhmatullin
dc1ee09481
Rename ASYNCIO_ENABLED to ASYNCIO_REACTOR, change the logic accordingly.
2019-12-27 21:56:28 +05:00
Andrey Rakhmatullin
20289be810
Merge remote-tracking branch 'origin/master' into asyncio-base
2019-12-20 19:31:20 +05:00
Marc Hernandez Cabot
23a67cec27
fix first letter capitalization for Raring and Scrapy
2019-12-19 09:57:17 +01:00
Marc Hernandez Cabot
c0d84f0962
fix typos
2019-12-18 19:39:21 +01:00
Andrey Rakhmatullin
3560123090
Rename ASYNCIO_SUPPORT to ASYNCIO_ENABLED.
2019-12-13 18:07:47 +05:00
Andrey Rakhmatullin
ed34ce14c0
Add the ASYNCIO_SUPPORT setting, reshuffle other logic accordingly.
2019-12-04 21:32:56 +05:00
Andrey Rahmatullin
229e722a03
Initial Python 2 removal ( #4091 )
2019-10-31 10:46:02 +01:00
Mikhail Korobov
8cb53441b0
Merge pull request #4056 from Gallaecio/documentation-build
...
Fix internal links in the tutorial and release notes
2019-10-19 00:00:02 +02:00
Bulat
84be6a941e
Refactor sentence.
2019-10-16 14:04:07 +03:00
Bulat
c9614a5bdd
Fixed BOT_NAME documentation
2019-10-16 12:07:19 +03:00
Adrián Chaves
a4aa5b8926
Fix internal links in the tutorial and release notes
2019-10-02 14:51:06 +02:00
Vostretsov Nikita
6bd88711f2
update documentation
2019-09-10 08:55:37 +00:00
Mikhail Korobov
ede91478e5
Merge pull request #3966 from anubhavp28/robotstxt_useragent
...
Adds ROBOTSTXT_USER_AGENT setting
2019-08-28 22:00:17 +05:00
Anubhav Patel
77c8ab2e62
makes suggested changes
2019-08-27 18:44:08 +05:30
Anubhav Patel
ad824a264b
fixes a link in doc
2019-08-27 18:30:11 +05:30
Anubhav Patel
00fe05e536
adds ROBOTSTXT_USER_AGENT setting
2019-08-19 09:24:16 +05:30
Andrey Rakhmatullin
aaa5229e5d
Fixes and improvements for DOWNLOADER_CLIENT_TLS_CIPHERS.
2019-08-13 16:56:26 +05:00
Andrey Rakhmatullin
ce281d890d
Documentation for DOWNLOADER_CLIENT_TLS_CIPHERS.
2019-08-13 16:53:19 +05:00
Anubhav Patel
8e813953bd
[MRG+1] [GSoC 2019] Interface for robots.txt parsers ( #3796 )
...
Make the robots.txt parser configurable through the new ROBOTSTXT_PARSER setting, support the Reppy and Robotexclusionrulesparser parsers, and allow implementing custom robots.txt parsers.
2019-08-02 09:43:29 +02:00
Mikhail Korobov
9c514b976f
Merge pull request #3450 from wRAR/tls-logging
...
Log cipher, certificate and temp key info on establishing an SSL connection
2019-07-24 01:45:40 +05:00
Mikhail Korobov
c679aefe85
Merge pull request #3660 from anubhavp28/logFormatter-doc-patch
...
[MRG+1] docs for scrapy.logformatter
2019-07-24 01:40:56 +05:00
Andrey Rakhmatullin
42743fd9dd
Move tls_verbose_logging extraction from __init__ to from_settings.
2019-07-18 20:49:25 +05:00
Adrián Chaves
a94b5bef3a
Write the 1.7 release notes and cover dropping Python 2 support in the upcoming 2.0
2019-07-18 15:37:17 +02:00
Andrey Rakhmatullin
98689b27a8
Improve the DOWNLOADER_CLIENTCONTEXTFACTORY doc.
2019-07-11 14:02:35 +05:00
Andrey Rakhmatullin
0b9dce3a6c
Add DOWNLOADER_CLIENT_TLS_VERBOSE_LOGGING setting.
2019-07-08 17:40:56 +05:00
Adrián Chaves
92fa657f9e
Merge pull request #3730 from ankostis/doc_logstats_delay
...
[MRG+1] doc: document LOGSTATS_INTERVAL setting
2019-04-11 17:34:45 +02:00
Kostis Anagnostopoulos
a8f83ab967
doc: document LOGSTATS_INTERVAL setting
2019-04-06 14:58:32 +02:00
Adrián Chaves
1ee99e1f42
Update docs/topics/settings.rst
...
Co-Authored-By: whalebot-helmsman <whalebot.helmsman@gmail.com>
2019-03-29 10:29:15 +00:00
Vostretsov Nikita
103fdc40c1
Merge remote-tracking branch 'upstream/master' into round-robin-scheduler-tested
2019-03-25 16:16:42 +03:00
Vostretsov Nikita
8afffb7234
Tests Cleanup
...
add doctest for function
no need in this variables
move common assertion inside function
rename variable
rename variables
rename function
use function
this is not a method of public API
correct name for test
Update docs/topics/settings.rst
Co-Authored-By: whalebot-helmsman <whalebot.helmsman@gmail.com>
2019-03-25 16:08:40 +03:00
Mikhail Korobov
83eb537645
assorted cleanups: comments, docstrings, etc
...
scheduler cleanup
Scheduler no longer converts requests to dicts; PriorityQueue
instances always work with Request instances; converting Requests
to dicts is now Priority Queue responsibility.
minor cleanup
2019-03-25 16:08:40 +03:00
Vostretsov Nikita
338b78d796
Add documentation
...
add section to broad-crawl topic
reword in accord with broad-crawl topic
add documentation for new priority queue
2019-03-22 15:40:37 +03:00
Anubhav Patel
66a502db8c
Merge branch 'master' into logFormatter-doc-patch
2019-03-15 00:28:54 +05:30
