Daniel Graña
dd473145f2
Merge pull request #1473 from ArturGaspar/master
...
[MRG+1] Support for returning deferreds in middlewares
2015-09-02 13:04:26 -03:00
Robert Weindl
833efbfcbd
Update tutorial.rst
...
Add missing "scrapy." prefix.
2015-09-02 20:35:23 +05:00
Artur Gaspar
dd83f6123d
Test for same response object in downloader middleware test.
2015-09-02 11:36:15 -03:00
Artur Gaspar
2748b38592
Test for not calling the download function when downloader middleware returns a response in process_request.
2015-09-02 11:15:37 -03:00
Artur Gaspar
1f4af9d81e
Test for robots.txt middleware for logger not being called when the request for robots.txt is ignored.
2015-09-02 10:33:53 -03:00
Artur Gaspar
668e5fd257
Test for robots.txt middleware for processing a request for which the robots.txt parser is ready.
2015-09-02 10:23:24 -03:00
Artur Gaspar
a6a629e707
Call actual error logger in robots.txt middleware tests.
2015-09-02 10:23:24 -03:00
Artur Gaspar
9ce9a293a6
Always check robots.txt before making another request in RobotsTxtMiddleware.
2015-09-02 10:23:24 -03:00
Daniel Graña
af0873746f
Merge pull request #1289 from pawelmhm/dont_retry_400
...
[MRG+1] [settings/default_settings.py] dont retry 400
2015-09-02 00:39:30 -03:00
Daniel Graña
3919ad64c5
Merge pull request #1470 from scrapy/py3-middlewares
...
[MRG] port some downloader middlewares to Python 3
2015-09-02 00:17:37 -03:00
Daniel Graña
bc499cb552
Merge pull request #1472 from olafdietsche/incomplete-submit-button
...
[MRG+1] Incomplete submit button
2015-09-02 00:16:42 -03:00
Artur Gaspar
ca83a0b028
Support for returning deferreds in downloader middleware methods.
2015-09-01 13:22:43 -03:00
Olaf Dietsche
937277e859
Make sure value attribute values are set.
2015-09-01 12:44:04 +02:00
Olaf Dietsche
2c28b53cc0
Add tests with incomplete buttons
2015-09-01 12:35:49 +02:00
Mikhail Korobov
c44cafe4f5
PY3 fix UserAgentMiddleware tests
2015-09-01 04:11:33 +05:00
Mikhail Korobov
3a9c73bc5d
PY3 fix DownloaderStats middleware tests
2015-09-01 04:09:15 +05:00
Mikhail Korobov
78a4cd0f1c
PY3 fix HttpAuthMiddleware tests
2015-09-01 04:00:26 +05:00
Mikhail Korobov
179a4409fb
PY3 enable DownloadTimeoutMiddleware tests
2015-09-01 03:58:25 +05:00
Mikhail Korobov
3cf1911a92
PY3 fix DefaultHeadersMiddleware tests
2015-09-01 03:57:05 +05:00
Mikhail Korobov
45101829a5
PY3 fix AjacCrawlable middleware tests
2015-09-01 03:49:52 +05:00
Daniel Graña
b876755f1c
Merge pull request #1469 from olafdietsche/form-submit-button-plain
...
Form submit button plain
2015-08-31 19:30:41 -03:00
Olaf Dietsche
121d7535be
Allow button cotrols to submit form
2015-08-31 19:58:16 +02:00
Olaf Dietsche
e5f26078fa
Add test: submit form with button control
2015-08-31 19:57:45 +02:00
Daniel Graña
0018caf0b6
Merge pull request #1461 from scrapy/py3-linkextractors
...
[MRG] assorted Python 3 porting
2015-08-31 11:59:30 -03:00
Daniel Graña
b4e382c867
Merge pull request #1318 from yarikoptic/fix-docs-explicit-rtd-theme-path
...
[MRG+1] DOC(ENH): specify path to rtd theme explicitly
2015-08-31 10:42:02 -03:00
Mikhail Korobov
44bfcbcf0f
TST split LinkExtractorTestCase.test_extraction into several methods; remove duplicated test
2015-08-31 00:49:38 +05:00
Mikhail Korobov
f7052413e0
PY3 raise an exception if bytes are passed as url to Link constructor
2015-08-28 23:04:02 +05:00
Mikhail Korobov
9bfe6ece59
Merge branch 'master' into py3-linkextractors
...
Conflicts:
scrapy/linkextractors/lxmlhtml.py
tests/test_linkextractors.py
2015-08-28 04:53:32 +05:00
Mikhail Korobov
ff24cbbc47
PY3 depth, offsite and referer spider middlewares; Crawler
2015-08-28 04:11:30 +05:00
Mikhail Korobov
d5984bbea9
PY3 port scrapy.spiders
2015-08-28 04:11:30 +05:00
Mikhail Korobov
f2edbd05de
PY3 port LinkExtractor
...
* tests for other link extractors are moved to test_linkextractors_deprecated.py
* in Python 3 Link is converted to use native strings for urls
* minor cleanups
2015-08-28 04:11:30 +05:00
Mikhail Korobov
f46a450080
refactor test_linkextractors
...
* rename LinkExtractorTestCase to BaseSgmlLinkExtractorTestCase
* add BaseLinkExtractorTestCase link extractor tests can inherit from
and decouple it from SgmlLinkExtractor
* add an extra check for deny_extensions
* xfail test_restrict_xpaths_with_html_entities for LxmlLinkExtractor explicitly
2015-08-28 04:11:30 +05:00
Julia Medina
aa31811cfd
Merge pull request #1214 from rgtk/link-rel
...
[MRG+1] Support link rel attribute with multiple values
2015-08-27 19:57:54 -03:00
Mikhail Korobov
9bfab53075
Merge pull request #1462 from tagatac/master
...
minor: scrapy.Spider docs grammar
2015-08-28 02:42:37 +05:00
David Tagatac
08162a15d8
minor: scrapy.Spider docs grammar
2015-08-27 17:37:16 -04:00
Rafał Gutkowski
cb3007c066
support link rel attribute with multiple values
2015-08-27 20:13:47 +02:00
Mikhail Korobov
8ecc4544b3
enable console tests in PY3
2015-08-27 21:37:24 +05:00
Mikhail Korobov
9616d91e4a
Merge pull request #1444 from cyberplant/bpython_support
...
[MRG +1] bpython support
2015-08-27 21:28:05 +05:00
Rick
aa88fd797a
Small grammatical change
2015-08-26 15:01:50 -03:00
Daniel Graña
71bd79e70f
Merge pull request #1212 from Digenis/test-fix-render_template_file
...
fix utils.template.render_templatefile() bug +test
2015-08-26 14:46:42 -03:00
Νικόλαος-Διγενής Καραγιάννης
56b3cf07ff
fix string manipulation in render_templatefile()
2015-08-26 20:45:41 +03:00
Νικόλαος-Διγενής Καραγιάννης
f5c54072ec
Test utils.template.render_templatefile()
2015-08-26 20:44:33 +03:00
Daniel Graña
026a1caffb
Merge pull request #1456 from scrapy/py3-fixes
...
Small Python 3 fixes
2015-08-26 11:46:00 -03:00
Mikhail Korobov
68a47ade46
PY3 port test_logformatter
2015-08-26 02:34:21 +05:00
Mikhail Korobov
642af00bb7
fix Referer logging
2015-08-26 02:19:33 +05:00
Mikhail Korobov
7da769feb2
enable test_stats and test_utils_log tests in Python 3
2015-08-26 01:58:59 +05:00
Mikhail Korobov
787b5af30e
add coverage files to gitignore
2015-08-26 01:58:33 +05:00
Mikhail Korobov
cfae62f9cc
Merge pull request #1441 from aivarsk/fix-common-practices
...
Make common practices sample code match the comments
2015-08-23 17:36:09 +05:00
Daniel Graña
489c76b83e
Merge pull request #1447 from jdemaeyer/fix/redirectmiddleware-respect-meta-attributes
...
[MRG +1] Fix RedirectMiddleware not honouring handle_httpstatus meta keys
2015-08-22 09:46:27 -03:00
Jakob de Maeyer
d164398a27
Fix RedirectMiddleware not honouring meta handle_httpstatus keys
2015-08-21 13:22:42 +02:00
