1
0
mirror of https://github.com/scrapy/scrapy.git synced 2025-03-15 11:10:38 +00:00

28 Commits

Author SHA1 Message Date
Eugenio Lacuesta
94d7ad76cb
Fix pycodestyle E2XX (whitespace) () 2020-04-15 14:11:37 +02:00
Adrián Chaves
6aa0ba4553
Write release notes for Scrapy 2.0.0 () 2020-03-03 09:11:11 +01:00
sakshamb2113
c411a51f42
Fix random failures from test_fixed_delay in some machines () 2020-02-28 13:17:02 +01:00
elacuesta
9d983c1b99
Expose certificate for HTTPS responses ()
* Expose certificate for HTTPS responses

* Fix test (missing inlineCallbacks decorator)

* Note about Response.certificate

* Explicitly cover None as the default value of Response.certificate

Co-authored-by: Adrián Chaves <adrian@chaves.io>
2020-02-22 17:20:31 +05:00
Andrey Rakhmatullin
36dcf90184 Also test non-default async callbacks. 2020-02-11 01:00:16 +05:00
Adrián Chaves
4c6993f264
Merge branch 'master' into asyncio-parse-asyncgen 2020-02-07 21:34:54 +01:00
Andrey Rakhmatullin
7323780c97 Support yield in async def callbacks. 2020-02-07 21:32:45 +05:00
Andrey Rakhmatullin
31f6c7112f Add a test for an async callbacks that returns requests. 2020-02-07 17:14:52 +05:00
Andrey Rakhmatullin
cc825c21de Test returning items from an async def callback. 2020-01-30 16:17:06 +05:00
Andrey Rakhmatullin
a91a13b443 Support for async def callbacks. 2020-01-09 15:59:19 +05:00
Mikhail Korobov
3bcb14fe4d
Merge pull request from Gallaecio/speed-up-tests-2
Improve the performance of the DOWNLOAD_DELAY test
2019-12-18 23:56:56 +05:00
elacuesta
916382e109 Add errback parameter to scrapy.spiders.crawl.Rule ()
* Add errback parameter to scrapy.spiders.crawl.Rule

* CrawlSpider: optimize by reducing iterations

* [test] Rule.errback

* [doc] Rule.errback

* [doc] Use autoclass in docs/topics/spiders.rst

Co-Authored-By: Adrián Chaves <adrian@chaves.io>

* Rule.process_links takes a list

* Fix aesthetic issue reported by Flake8
2019-12-18 20:05:33 +05:00
Adrián Chaves
058bdda0af Improve the performance of the DOWNLOAD_DELAY test 2019-11-14 16:53:20 +01:00
Eugenio Lacuesta
5ab0f189ce
Remove six.moves occurrences from tests 2019-11-03 00:01:09 -03:00
Vostretsov Nikita
b364d27247 [MRG+1] Automatic port selection for servicies in unit tests ()
* ability to pass port as a parameter

* try to find free ports

* use environment variables to pass mock server address

* get mock server address from environment variables

* ability to select ports for proxy in runtime

* use common method for URLs from mock server

* https support

* get mock server address

* get mock address

* replace hand-written mechanism by kernel-based one

* use ephemeral ports in mockserver

* strip EOL from addresses

* use ephemeral port in proxy

* no need to restore environment as it is restored in tearDown

* decode bytes

* use mockserver address as a variable

* ability to pass address as variable

* per test-case mockserver

* use base class

* remove obsolete environment manipulation

* return usage of proxy for http cases

* common method for broking proxy auth credentials

* python version-independent url methods
2018-05-23 21:25:50 +03:00
Paul Tremberth
02e1d2b1fd Add trailing dot 2017-02-02 22:28:37 +01:00
Paul Tremberth
e604c0f3ab Remove unused imports 2017-02-02 18:26:41 +01:00
Paul Tremberth
3358254c5c Make DNS retry test compatible with Twisted 17+ 2017-02-02 17:53:28 +01:00
Paul Tremberth
27456996a9 Add assertion on crawler not running 2016-11-08 11:46:16 +01:00
Paul Tremberth
61efacdd1f Add testcase for catching exception from open_spider() from pipeline 2016-11-08 11:35:42 +01:00
Konstantin Lopuhin
0680950b98 py3: pass CrawlTestCase.test_referer_header, fixing Echo resource in mockserver and json decoding in test 2016-01-21 17:52:12 +03:00
Mikhail Korobov
a49c82ad62 TST improve CrawlerRunner tests
* use CrawlerRunner.create_crawler instead of get_crawler helper in test_crawl;
* add a test for loading spiders by name;
* add a test for passing Crawler objects instead of Spider objects;
* add a test for CrawlerRunner.join
2015-10-30 20:24:37 +05:00
Jakob de Maeyer
8307c1212f Add ExecutionEngine.close() method 2015-10-29 10:03:29 +01:00
nyov
6e762ce25c PY3 renames (six types) 2015-08-08 04:21:58 +05:00
Julia Medina
c2d716807a Use LogCapture in testfixtures package for tests
This allows to remove `get_testlog` helper, `flushLoggedErrors` from
twisted.trial.unittest.TestCase and Twisted log observers created for
each test on conftest.py.
2015-04-22 17:24:40 -03:00
Berker Peksag
31e5f164d4 Import unittest.mock if available.
mock is in the stdlib since Python 3.3.
2015-03-08 15:03:04 +02:00
Julia Medina
870438e5f4 Update tests utils, fixing get_crawler and removing docrawl 2014-08-12 14:02:56 -03:00
Daniel Graña
242c08553f move tests under root dir 2014-07-30 19:45:08 -03:00