Konstantin Lopuhin
96fcf4cea4
add a check that byte url is not accepted in http.Response on py3
2016-01-15 13:27:28 +03:00
Konstantin Lopuhin
99f1f2ad1d
unskip tests and modules ported to py3
2016-01-14 19:00:48 +03:00
Konstantin Lopuhin
81a90c3af6
unskip part of test_download_gzip_response on py3, file a twisted issue for the remaining part
2016-01-14 18:47:06 +03:00
Konstantin Lopuhin
2aa6c92ffc
py3 fixes in tests.mockserver
2016-01-14 17:52:50 +03:00
Konstantin Lopuhin
f46a9d595d
skip ftp tests on py3 for now
2016-01-14 17:31:58 +03:00
Konstantin Lopuhin
4950f5988e
py3: pass http proxy tests
2016-01-14 17:24:08 +03:00
Konstantin Lopuhin
c6f14a39de
py3: fix http10 downloader - unicode host expected here
2016-01-14 16:50:16 +03:00
Konstantin Lopuhin
6b79fffa9a
py3: pass all of HttpTestCase
2016-01-14 16:37:18 +03:00
Konstantin Lopuhin
3509378b8b
py3: pass first http downloader test, simple crawler works now, yay!
2016-01-14 16:29:19 +03:00
Konstantin Lopuhin
32bb5b682a
fix import of test_downloader_handlers.py: use @implementer, move failing on py3 imports into corresponding tests
2016-01-14 16:11:16 +03:00
Konstantin Lopuhin
5c2241ccc7
py3: fix webclient tests after making ScrapyHTTPClientFactory use bytes as in twisted
2016-01-14 15:30:28 +03:00
Konstantin Lopuhin
6a412d2503
all tests pass in test_webclient.py on py3 - removing from py3-ignores
2016-01-14 13:48:48 +03:00
Konstantin Lopuhin
0178356178
py3 fix testRedirect: url is bytes here
2016-01-14 13:46:46 +03:00
Konstantin Lopuhin
30c7b4e4cc
py3 compat in test_timeoutTriggering cleanup
2016-01-14 13:44:14 +03:00
Konstantin Lopuhin
88f55312af
py3 fix in testFactoryInfo - factory attirbutes are bytes in twisted
2016-01-14 13:43:37 +03:00
Konstantin Lopuhin
b5f9bc8499
py3 test fixes in test_webclient - expect bytes as page body
2016-01-14 13:04:45 +03:00
Konstantin Lopuhin
ae4aa2c3b2
py3 test fix: putChild expects bytes as path
2016-01-14 13:04:10 +03:00
Konstantin Lopuhin
325b6af6c2
fix ScrapyHTTPPageGetterTests for py3 - we expect bytes here
2016-01-14 12:29:59 +03:00
Konstantin Lopuhin
945674eb8f
pass test_externalUnicodeInterference - the logic for py3 is clearly inverse of what was expected in this test, as scrapy Request url must be unicode
2016-01-14 12:25:54 +03:00
Konstantin Lopuhin
1d5ab67183
pass test_getPageHead on py3
2016-01-14 12:04:26 +03:00
Konstantin Lopuhin
9f2be23a39
webclient tests, py3: fix setUp, pass test_getPage
2016-01-14 11:42:23 +03:00
Paul Tremberth
d4872940db
PY3: port utils/iterators
2016-01-07 10:51:27 +01:00
Elias Dorneles
7d187735ff
Merge pull request #1359 from nyov/nyov/drop-optional_features
...
[MRG+1] drop deprecated "optional_features" set
2015-12-04 14:08:44 -02:00
Daniel Graña
17aba44f16
Merge pull request #1565 from starrify/fix-1564
...
[MRG+1] fixed: Issue #1564 (Incorrectly picked URL in `scrapy.linkextractors.regex.RegexLinkExtractor` when there is a `<base>` tag. )
2015-12-04 10:31:13 -03:00
Daniel Graña
dc65026395
Merge pull request #1597 from Digenis/various_formreq_fixes
...
[MRG+1] Various FormRequest tests+fixes
2015-12-04 09:55:12 -03:00
palego
2c251000d3
custom project templates
...
allow override of TEMPLATES_DIR for startproject
copy full TEMPLATES_DIR/project tree
doc update
2015-12-01 03:07:27 +01:00
Νικόλαος-Διγενής Καραγιάννης
395ef805eb
FormRequest: test unicode xpath expr & exception
2015-11-13 18:54:02 +02:00
Νικόλαος-Διγενής Καραγιάννης
7a438a51b7
FormRequest: test default <input> type (is text)
2015-11-13 18:03:54 +02:00
Νικόλαος-Διγενής Καραγιάννης
881bf19e68
FormRequest: test case-insensitive type attribute
2015-11-13 17:57:25 +02:00
Julia Medina
54216d7afe
Merge pull request #1586 from jdemaeyer/fix/backwards-compatible-per-key-priorities
...
[MRG+1] Backwards compatible per key priorities
2015-11-11 14:08:50 -03:00
Jakob de Maeyer
44f6ada034
Overwrite, not update, dictionary-like settings
2015-11-11 17:40:42 +01:00
Jakob de Maeyer
52ecee6a62
Replace BaseSettings._getcomposite() with public .getwithbase() method
2015-11-11 17:39:57 +01:00
Jakob de Maeyer
b6a023ce98
Add backwards compatibility for build_component_list
2015-11-11 17:39:56 +01:00
Mikhail Korobov
9548691fdd
Merge pull request #1563 from starrify/master
...
[MRG+1] fixed: Issue #1562 (Incorrectly picked URL in `scrapy.http.FormReques t.from_response` when there is a `<base>` tag)
2015-11-10 21:33:24 +05:00
Pengyu CHEN
94486bb294
added: Test case for the fix.
2015-11-02 23:00:42 +08:00
Pengyu CHEN
e19bf4aecc
added: Test case for the fix
2015-11-02 22:52:41 +08:00
Julia Medina
57f87b95d4
Merge pull request #1528 from scrapy/create-crawler
...
public Crawler.create_crawler method
2015-10-30 12:36:35 -03:00
Mikhail Korobov
a49c82ad62
TST improve CrawlerRunner tests
...
* use CrawlerRunner.create_crawler instead of get_crawler helper in test_crawl;
* add a test for loading spiders by name;
* add a test for passing Crawler objects instead of Spider objects;
* add a test for CrawlerRunner.join
2015-10-30 20:24:37 +05:00
Jakob de Maeyer
32ff4cc1d7
PEP8ify scrapy.settings tests
2015-10-29 19:52:19 +01:00
Jakob de Maeyer
51ca84c9b4
Add missing tests for scrapy.settings module
2015-10-29 17:34:18 +01:00
Jakob de Maeyer
8307c1212f
Add ExecutionEngine.close() method
2015-10-29 10:03:29 +01:00
Jakob de Maeyer
03f1720afb
Fix backwards-compatibility for users who explicitly set _BASE settings
2015-10-27 14:22:56 +01:00
Jakob de Maeyer
f249b309ab
Move scrapy.utils.conf.remove_none_values to s.u.python.without_none_values
2015-10-27 12:43:33 +01:00
Jakob de Maeyer
26586ef5a6
Deprecate _BASE settings, unify _BASE backwards-compatibility
2015-10-27 12:43:23 +01:00
Jakob de Maeyer
a769a1ef78
Introduce BaseSettings with full dictionary interface
2015-10-27 12:38:52 +01:00
Mikhail Korobov
215905bdb6
PY3 port bench, startproject, genspider, list and runspider commands
2015-10-12 17:59:49 +05:00
Νικόλαος-Διγενής Καραγιάννης
d66efb13ba
test xml nodename with dots
2015-10-07 14:43:47 +03:00
preetwinder
47c8e2ba78
Restructure tests for add_http_if_no_scheme function
2015-09-24 17:57:25 +00:00
preetwinder
8c629eee3e
adds docstring, tests and correction
2015-09-24 17:28:44 +00:00
preetwinder
c517951a48
add_scheme_if_missing for scrapy shell command
2015-09-24 17:28:44 +00:00
