1
0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-22 06:13:24 +00:00

7679 Commits

Author SHA1 Message Date
Andrey Rakhmatullin
30ebd05a5f Simplify the tox asyncio entries. 2019-12-27 00:05:14 +05:00
Andrey Rakhmatullin
37ac47ff80 Fix a deprecation warning. 2019-12-26 20:46:54 +05:00
Andrey Rakhmatullin
87ece066ca Remove conditional asyncio imports. 2019-12-26 20:41:06 +05:00
Andrey Rakhmatullin
8de80f59db Raise an exception if ASYNCIO_ENABLED but the reactor is wrong. 2019-12-21 13:08:29 +05:00
Andrey Rakhmatullin
e342de5038 Remove a stray newline. 2019-12-20 19:37:50 +05:00
Andrey Rakhmatullin
40697dcbfa Remove deferred_from_coro from this PR. 2019-12-20 19:33:44 +05:00
Andrey Rakhmatullin
20289be810 Merge remote-tracking branch 'origin/master' into asyncio-base 2019-12-20 19:31:20 +05:00
Mikhail Korobov
fb3fb17c28
Merge pull request #4045 from Gallaecio/documentation-coverage
Deprecate FilteringLinkExtractor direct usage; add API docs for scrapy.linkextractors
2019-12-20 02:01:45 +05:00
Mikhail Korobov
6a32a36e09
Merge pull request #4143 from Gallaecio/undent-rst-doctests
Do not indent doctests from the documentation unnecessarily
2019-12-20 01:59:07 +05:00
Mikhail Korobov
1213c0e5f0
Merge pull request #4247 from noviluni/fix_typos
fix typos
2019-12-20 01:56:36 +05:00
Mikhail Korobov
a611a7c0f2
Merge pull request #4249 from Gallaecio/docs-py37
Use Python 3.7 to build the documentation
2019-12-20 01:54:26 +05:00
Adrián Chaves
b4a2d985a7
Merge branch 'master' into undent-rst-doctests 2019-12-19 12:39:17 +01:00
Adrián Chaves
e22c0c27d9 Revert "Improve FilteringLinkExtractor.__new__"
This reverts commit ee9881d2704798c9cd61b6da503bb0694227c58c.
2019-12-19 12:15:54 +01:00
Adrián Chaves
f6bc1940a3 Use Python 3.7 to build the documentation 2019-12-19 12:06:15 +01:00
Andrey Rahmatullin
c841a1f3e7
Merge pull request #4140 from Gallaecio/docs-py38
Use the latest Python version to build the documentation
2019-12-19 14:09:02 +05:00
Marc Hernandez Cabot
16b363de31 Merge branch 'master' into fix_typos 2019-12-19 10:02:21 +01:00
Marc Hernandez Cabot
23a67cec27 fix first letter capitalization for Raring and Scrapy 2019-12-19 09:57:17 +01:00
Mikhail Korobov
3bcb14fe4d
Merge pull request #4164 from Gallaecio/speed-up-tests-2
Improve the performance of the DOWNLOAD_DELAY test
2019-12-18 23:56:56 +05:00
Mikhail Korobov
8f7faaa63d
Merge pull request #4190 from Gallaecio/doctest
Make developer-tools doctests pass
2019-12-18 23:40:21 +05:00
Marc Hernandez Cabot
c0d84f0962 fix typos 2019-12-18 19:39:21 +01:00
Mikhail Korobov
c976230b5f
Merge pull request #4246 from noviluni/fix_W293_blank_line_contains_whitespace
Fix W291, W292 and W293 flake8 issues (whitespaces)
2019-12-18 23:39:04 +05:00
Marc Hernandez Cabot
a5de2c64e6 fix W291, W292, W293 (whitespaces) 2019-12-18 16:33:46 +01:00
elacuesta
916382e109 Add errback parameter to scrapy.spiders.crawl.Rule (#4000)
* Add errback parameter to scrapy.spiders.crawl.Rule

* CrawlSpider: optimize by reducing iterations

* [test] Rule.errback

* [doc] Rule.errback

* [doc] Use autoclass in docs/topics/spiders.rst

Co-Authored-By: Adrián Chaves <adrian@chaves.io>

* Rule.process_links takes a list

* Fix aesthetic issue reported by Flake8
2019-12-18 20:05:33 +05:00
Andrey Rahmatullin
0e8ee22a88
Merge pull request #4244 from wRAR/split-EngineTest
Split a long test in test_engine.py into three.
2019-12-18 20:00:36 +05:00
Adrián Chaves
00517ec273
Merge branch 'master' into docs-py38 2019-12-18 15:50:49 +01:00
Andrey Rakhmatullin
7ccb169a27 Split a long test in test_engine.py into three. 2019-12-18 19:41:16 +05:00
Mikhail Korobov
053319334c
Merge pull request #4179 from Gallaecio/user-friendlier-tox
Make tox configuration more user friendly
2019-12-18 18:35:19 +05:00
Andrey Rahmatullin
e2b5cdeb88
Merge pull request #4242 from whalebot-helmsman/single_place_for_dependencies
Remove requirements-py3.txt
2019-12-18 18:27:09 +05:00
Vostretsov Nikita
012533924a remove requirements from here too 2019-12-18 11:13:36 +00:00
Mikhail Korobov
e90f276903
Merge pull request #4198 from wRAR/deprecate-noconnect
Deprecate the HTTPS proxy noconnect mode.
2019-12-18 16:11:24 +05:00
Adrián Chaves
1f689b50c6 Merge remote-tracking branch 'origin/documentation-coverage' into documentation-coverage 2019-12-18 12:09:34 +01:00
Adrián Chaves
174769a3f0 Use a better name for the LxmlLinkExtractor subclassing test 2019-12-18 12:09:03 +01:00
Adrián Chaves
ee9881d270 Improve FilteringLinkExtractor.__new__ 2019-12-18 12:08:34 +01:00
Vostretsov Nikita
12f9ffeb5d remove requirements-py3.txt 2019-12-18 10:53:27 +00:00
Andrey Rakhmatullin
ac302c3f61 Fix a flake8 problem. 2019-12-18 15:43:05 +05:00
Andrey Rakhmatullin
bb2ff13e4c Skip Http10ProxyTestCase.test_download_with_proxy_https_noconnect 2019-12-18 15:39:08 +05:00
Andrey Rakhmatullin
2d92a39003 Restore test_download_with_proxy_https_noconnect, check for a warning there. 2019-12-18 12:07:08 +05:00
Adrián Chaves
607815dfc2
Merge branch 'master' into documentation-coverage 2019-12-17 16:54:27 +01:00
Adrián Chaves
b8cf522916
Merge branch 'master' into user-friendlier-tox 2019-12-17 16:53:08 +01:00
Andrey Rahmatullin
4be2bbfe75
Merge pull request #4239 from Apuyuseng/master
Fix mail attachs tcmime *** (#4229)
2019-12-17 19:43:30 +05:00
Andrey Rahmatullin
bb3f164280
Merge pull request #4236 from wRAR/pipeline-tests
Add simple tests for pipelines.
2019-12-17 19:38:06 +05:00
Marc Hernández
63cf5c75c8 Fix E502: backslash is redundant between brackets (#4238) 2019-12-17 13:53:15 +01:00
apu
7d0096da6e
Fix mail attachs tcmime *** (#4229)
When the file name consists of alphanumeric characters, it is normal to receive the attachment name.
However,However, problems will occur if the file name is changed to Chinese.
This has nothing to do with the file type
2019-12-17 09:47:01 +08:00
Andrey Rakhmatullin
5980d3bbff Add simple tests for pipelines. 2019-12-16 22:41:28 +05:00
Mikhail Korobov
e9b24d62c6
Merge pull request #4231 from noviluni/docs_fix
Docs fix
2019-12-16 21:56:43 +05:00
Andrey Rakhmatullin
900de7c146 Fix the reactor_pytest fixture. 2019-12-16 21:11:58 +05:00
Andrey Rakhmatullin
039e6fe691 Refactor install_asyncio_reactor slightly. 2019-12-16 20:17:41 +05:00
Andrey Rakhmatullin
2db7d45378 Enable skipping tests based on --reactor. 2019-12-16 19:25:49 +05:00
marc
a59bb279d1 add year through code 2019-12-15 17:33:00 +01:00
marc
1aab20e1ce update copyright notice year 2019-12-14 10:37:31 +01:00