1
0
mirror of https://github.com/scrapy/scrapy.git synced 2025-03-14 20:53:10 +00:00

2446 Commits

Author SHA1 Message Date
Matthew Donoughe
40d9ca3bdd
use pathlib 2022-10-17 17:40:10 -04:00
Andrey Rakhmatullin
eeb199adda Fix flake8 issues in previously ignored files. 2022-10-10 11:10:59 +06:00
Mikhail Korobov
4f296b61b0
Merge pull request from zaid-ismail031/master
Changed incorrect info in documentation regarding the return type of parse method.
2022-10-03 00:19:08 +05:00
Adrián Chaves
1d79994dcc Copy 2.6.3 release notes from the 2.6 branch 2022-09-27 17:01:33 +02:00
Gábor Lipták
79a4bc3da0
Cleanup METHOD_SSLv3
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
2022-09-25 17:01:43 -04:00
Tim B
5f19420211
Documented how settings must be picklable ()
Co-authored-by: Adrián Chaves <adrian@chaves.io>
Co-authored-by: Laerte Pereira <5853172+Laerte@users.noreply.github.com>
2022-09-21 08:27:27 +02:00
Gábor Lipták
1429aa011c
Update test-standard link in contributing docs () 2022-09-20 18:47:20 +02:00
Magsen (CD)
ce0ca51485
fix: typo in tutorial
fix: typo in tutorial
2022-09-13 12:07:58 +02:00
zaid-ismail031
c49b5aaf77 Changed incorrect information regarding the return type of parse/request callback method. 2022-08-26 21:17:31 +02:00
Andrey Rahmatullin
0f1112f3e2
Update docs/index.rst
Co-authored-by: Adrián Chaves <adrian@chaves.io>
2022-07-27 23:12:43 +05:00
Andrey Rahmatullin
83ecdf1bca
Update docs/topics/components.rst
Co-authored-by: Adrián Chaves <adrian@chaves.io>
2022-07-27 23:12:31 +05:00
Mikhail Korobov
4dde2f2c36 Merge branch 'master' into asyncio-parse-fixes 2022-07-27 09:32:54 +05:00
Adrián Chaves
00ccb02f1f Merge remote-tracking branch 'upstream/2.6' 2022-07-25 14:13:22 +02:00
Adrián Chaves
af7dd16d8d
Merge pull request from GHSA-9x8m-2xpf-crp3
* Enforce matching proxy request meta and Proxy-Authorization header

* Cover proxy credential security fix in the release notes

* Remove extra empty line

* Reword the security issue description

* Address scenario where Proxy-Authorization is unexpectedly removed by a prior middleware

* Set the release date of Scrapy 2.6.2 and 1.8.3
2022-07-25 13:15:17 +02:00
Andrey Rahmatullin
a0681fb811
Merge branch 'master' into asyncio-parse-asyncgen-proper-rebased 2022-07-22 18:30:38 +05:00
Mikhail Korobov
4205609051 Fixed intersphinx references 2022-07-21 18:02:56 +02:00
Mikhail Korobov
f60c7ae768
Fixed heading levels in downloader middleware docs () 2022-07-20 11:01:22 +02:00
Mikhail Korobov
9b33b82a8b Fixed intersphinx references 2022-07-17 15:50:40 +05:00
Mikhail Korobov
77a3b02523
Merge pull request from Rotzbua/fix-doc-typo
Fix doc: `scrapy.exporter` to `scrapy.exporters`
2022-07-17 00:21:50 +05:00
Rotzbua
1c7ed4f2e5
[doc] Remove incompatible web service project
* Abandoned since 2017
* Not compatible with Python3
2022-07-12 13:09:20 +02:00
Rotzbua
09c3a4ad08
Fix doc: scrapy.exporter to scrapy.exporters 2022-07-12 12:45:15 +02:00
Mikhail Korobov
500dae82e2
Merge pull request from Gallaecio/allow-customizing-export-column-names
Allow customizing export column names
2022-06-27 00:26:05 +05:00
Emanuele
d8223adfac
Typo: cleanup (verb) → clean up () 2022-06-20 11:54:05 +02:00
Adrián Chaves
e3e69d1209
Pin documentation requirements () 2022-06-20 11:46:13 +02:00
Adrián Chaves
54bfb9649b
Cover in the 2.6.2 release notes () 2022-06-17 11:51:02 +02:00
Adrián Chaves
4ef71829b2
If TWISTED_REACTOR is None, reuse any pre-installed reactor () 2022-06-17 10:37:27 +02:00
Michel Ace
6e878490e8
Support and prefer the .jsonl file extension () 2022-06-17 08:37:14 +02:00
Adrián Chaves
3e994bda45 Update for Python 3.7+ 2022-06-16 20:53:14 +02:00
Adrián Chaves
e8503217fb Merge remote-tracking branch 'upstream/master' into allow-customizing-export-column-names 2022-06-16 20:19:52 +02:00
Mikhail Korobov
de0e2ccd7b
Merge pull request from FJMonteroInformatica/master
make test files follow HTML conventions
2022-06-12 00:46:45 +05:00
Laerte Pereira
197aca2c94
docs: Remove leftover 2022-06-09 10:11:49 -03:00
Laerte Pereira
c4c5c9f258
docs: Remove minimal versions paragraphs 2022-06-09 10:00:44 -03:00
Laerte Pereira
2e6721fd86
docs: Update minimal versions that Scrapy is tested against 2022-06-09 08:37:01 -03:00
Laerte Pereira
407562b38b
Drop Python 3.6 support ()
* chore: Drop Python 3.6 support

* Attend PR comments

* Tweak versions

* Update dependencies version

* fix: Ubuntu workflow

* fix windows workflow

* chore: Remove comment

* update `install_requires` dependencies versions

* move lxml to main pinned requirements

* Attend code-review comments

* remove non-pinned 3.7 from windows workflow

* simplify condition

* lint

* remove paragraph

* refactor

* remove leftover
2022-06-09 00:25:03 -03:00
Adrián Chaves
c5627af15b
Centralize request fingerprints ()
Co-authored-by: Mikhail Korobov <kmike84@gmail.com>
2022-06-07 18:44:54 +02:00
Mikhail Korobov
517cbc8d71
Merge pull request from mtabbasi/thumb_path-access-item
pass on item to thumb_path function as additional argument
2022-06-05 03:42:45 +05:00
Mikhail Korobov
bafe874ecc
Merge pull request from burakcank/signal_coroutine
Document coroutines for signals.
2022-05-28 13:38:05 +05:00
Mohammadtaher Abbasi
f39def4492 add docs 2022-05-25 23:57:38 +04:30
Adrián Chaves
1c1cd5d8ea Update the 2.6.2 release notes 2022-05-20 07:05:26 +02:00
Adrián Chaves
84c29a286f
Unset the release date of still-unreleased 2.6.2 () 2022-05-20 06:45:38 +02:00
Adrián Chaves
aead27bcbd
Add release notes for 2.6.2 () 2022-04-14 15:06:22 +02:00
Yash
319e67f779 documentation update for multiple spiders
i noticed passing settings to configure logging function made weird output go away. checked documentation and it says first parameter is settings file. Is this correct?
2022-04-12 11:42:22 -05:00
Adrián Chaves
bae3f87455 Cover a backward-incompatible Request serialization change in the 2.6 release notes 2022-04-08 12:04:02 +02:00
Andrey Rahmatullin
8c3e5a2421
Merge pull request from d00399830/typo_fixes_sm
Fix capitalization of JavaScript within the Developer Tools documentation
2022-03-23 11:06:54 +05:00
Víctor Ruiz
2227be7af6
Fix a typo in the HTTP cache documentation () 2022-03-22 15:21:16 +01:00
D00399830
fcf3d8e0a0 Updated the documentation for developer tools to have JavaScript instead of Javascript, as JavaScript is the more correct way to write it 2022-03-21 14:09:31 -06:00
Adrián Chaves
b95c634b86 Document how to enforce Scrapy versions on Scrapy components 2022-03-17 22:23:25 +01:00
FJMonteroInformatica
6a3f2ee687 HTML Conventions 2022-03-17 20:09:56 +01:00
Eugene
9a28eb0bad
Suggest installing the brotli package instead of brotlipy () 2022-03-17 05:39:54 +01:00
Adrián Chaves
b78e6915c6 Clarify that without async-to-sync conversions items yielded before an exception are processed 2022-03-16 20:17:25 +01:00