Paul Tremberth
a19dfafb25
Support case-insensitive domains in url_is_from_any_domain()
2014-04-16 22:33:30 +02:00
Mikhail Korobov
e8e689acd1
fix scrapy.version_info when SCRAPY_VERSION_FROM_GIT is set
2014-04-16 22:59:28 +06:00
Mikhail Korobov
c3426e952b
TST fix tests that became broken after adding top-level imports and switching to py.test.
...
py.test replaces AssertionError exception with its own subclass in an import hook.
For some reason when top-level shortcuts were added "except AssertionError" in
contracts.__init__ meant "except py.test AssertionError" while scrapy.exceptions.ContractFail
was still a subclass of a builtin AssertionError, so Contracts reported errors
instead of failures and this made tests in test_contracts.py fail.
For now - just disable fancy py.test asserts.
2014-04-15 06:54:14 +06:00
Mikhail Korobov
11e62117b7
suggest scrapy.Selector in deprecation warnings
2014-04-15 01:09:35 +06:00
Mikhail Korobov
2d3803672b
DOC use top-level shortcuts in docs
2014-04-15 01:09:35 +06:00
Mikhail Korobov
4ddad88d6f
use "import scrapy" in templates
2014-04-15 01:09:35 +06:00
Daniel Graña
62c7daf785
Add basic top-level shortcuts
2014-04-15 01:09:35 +06:00
Daniel Graña
9a643d689c
cleanup toplevel namespace
2014-04-15 01:09:35 +06:00
Mikhail Korobov
7814cbf319
remove unused import
2014-04-15 00:49:47 +06:00
Daniel Graña
2c8eceb5b8
Merge pull request #659 from denysbutenko/master
...
[MRG] Resolved issue #546 . Output format parsing from filename extension.
2014-04-14 14:08:05 -04:00
Daniel Graña
0a7e655948
Merge pull request #681 from kmike/version_info
...
[MRG] make scrapy.version_info a tuple of integers
2014-04-14 13:39:27 -04:00
Irhine
86a60e2cfc
support i18n by using utf-8 coding template files
2014-04-14 18:14:29 +08:00
Irhine
943b6f90e9
add encoding utf-8 to the first line
2014-04-14 17:05:42 +08:00
Daniel Graña
21bff7b3b3
Merge pull request #687 from Curita/docs-linkcheck
...
Docs build and linkcheck in tox env
2014-04-11 15:36:00 -04:00
Julia Medina
80081054a2
Fix broken links in documentation
2014-04-09 18:57:52 -03:00
Julia Medina
f0a0d0bc22
Ignore known broken links in docs linkcheck
2014-04-09 18:46:21 -03:00
Julia Medina
c7c57178e0
New tox env: docs
2014-04-09 18:46:01 -03:00
Daniel Graña
96ef4c603c
Merge pull request #679 from Curita/pytest
...
Run tests with pytest instead of trial
2014-04-08 10:41:03 -07:00
Daniel Graña
e71b8f266d
Merge pull request #676 from csalazar/fix_xxe_flaw
...
Fixed XXE flaw in sitemap reader
2014-04-08 10:40:04 -07:00
Julia Medina
dd9d3af9b3
Ensure spiders module reload between spider manager tests
2014-04-04 21:47:47 -03:00
Julia Medina
c50f088b63
Fix httpcache doctest that assumed dictionary order
2014-04-04 21:47:47 -03:00
Julia Medina
1186969715
Change function name so it does not mess up with pytest autodiscover
2014-04-04 21:47:47 -03:00
Julia Medina
81187feaac
Ignore files with import errors on pytest test discover
2014-04-04 21:47:19 -03:00
Julia Medina
560a84657c
Support doctest and __init__.py test discover in pytest
2014-04-04 21:43:12 -03:00
Julia Medina
1779b31e9b
Add py33 environment to allowed failures in travis-ci
2014-04-04 21:43:12 -03:00
Julia Medina
20b4c8f3ea
Trial functionality for running tests with pytest
...
* Change current dir to tmp dir on each test run
* Log twisted with test.log
2014-04-04 21:42:00 -03:00
Mikhail Korobov
713aec3919
make scrapy.version_info a tuple of integers
2014-04-05 05:54:49 +06:00
Claudio Salazar
554102fd70
Added resolve_entities to kwargs in SafeXMLParser
2014-04-05 00:40:41 +08:00
Claudio Salazar
d034df36e8
Added test against XXE attacks for Sitemap
2014-04-05 00:22:36 +08:00
Claudio Salazar
c2a424daae
Fixed XML selector against XXE attacks
2014-04-05 00:13:27 +08:00
Daniel Graña
986be9a396
Run testsuite with py.test
2014-04-03 12:26:44 -03:00
Daniel Graña
f3d15085b4
Merge pull request #674 from dangra/pypy
...
[WIP] try to restore pypy tests
2014-04-02 19:45:00 -07:00
Claudio Salazar
43217fd698
Fixed XXE flaw in sitemap reader
2014-04-01 23:54:04 +08:00
Daniel Graña
95853b325b
try to restore pypy tests
2014-03-28 15:56:23 -03:00
Daniel Graña
ade7662f23
Merge pull request #661 from ananana/sgml-attrs-tuple
...
Fixed default value of attrs argument in SgmlLinkExtractor to be tuple
2014-03-27 09:06:43 -07:00
Ana Sabina Uban
73109bf950
Fixed SgmlLinkExtractor constructor to properly handle both string and list parameters (attrs, tags, deny_extensions)
2014-03-27 16:12:00 +02:00
Denys Butenko
7e353761a3
Added more verbose error message for unrecognized output format. PEP8.
2014-03-26 12:27:52 +02:00
Daniel Graña
f687455046
Merge pull request #660 from rubenvereecken/issue193
...
Added content-type check as per issue #193
2014-03-24 06:10:23 -07:00
Ruben Vereecken
54434095fc
Redefined test for #193
2014-03-21 14:41:07 +01:00
Paul Brown
a1ee354609
fixed typo
2014-03-20 15:16:48 -05:00
Ruben Vereecken
92780b70da
Added content-type check as per issue #193
2014-03-20 20:41:08 +01:00
Denys Butenko
aa79abc5bd
Add import os for crawl.
2014-03-20 12:36:05 +02:00
Denys Butenko
e64d6e3f3c
Fix default value.
2014-03-19 21:32:32 +02:00
Denys Butenko
9822eb78cc
Added back -t option. If --output-format not defined parse from extension --output
2014-03-19 21:18:01 +02:00
Denys Butenko
8e0b2bd343
Resolved issue #546 . Output format parsing from filename extension.
2014-03-19 19:00:55 +02:00
Daniel Graña
3df6946256
Merge pull request #656 from kmike/httperror-logging
...
[MRG] Fix for #612 + small tests cleanup
2014-03-18 19:46:59 -07:00
Mikhail Korobov
60f9dc7a54
TST fix file descriptor leak and a bad variable name in get_testlog
2014-03-19 04:18:04 +06:00
Mikhail Korobov
75060d1424
Fix for #612 + integration-style tests for HttpErrorMiddleware
2014-03-19 04:04:24 +06:00
Mikhail Korobov
364790eb07
scrapy.utils.test.docrawl function
2014-03-19 03:30:58 +06:00
Mikhail Korobov
15454e7845
reduce code duplication in test_spidermiddleware_httperror
2014-03-19 01:50:09 +06:00
