Eugenio Lacuesta
|
9aea1f0961
|
Remove backslash (tests)
|
2020-07-09 11:04:46 -03:00 |
|
Eugenio Lacuesta
|
182394bcec
|
E501 compliance (tests)
|
2020-03-22 23:29:30 -03:00 |
|
nyov
|
3b0820d747
|
Deprecate Spider.make_requests_from_url, part 2 (#4412)
|
2020-03-12 20:15:49 +01:00 |
|
Mikhail Korobov
|
250da28952
|
Merge pull request #4170 from mabelvj/4133-handle-start_url
Raise error when start_url found instead of start_urls.
|
2019-12-05 17:47:03 +05:00 |
|
Mabel Villalba
|
070b3a4e84
|
Merge branch 'master' into 4133-handle-start_url
|
2019-11-21 17:10:31 +01:00 |
|
Mabel Villalba
|
1718e450ef
|
[start_url] Fixes #4133: Raise AttributeError error when empty 'start_urls' and 'start_url' found. Added test.
|
2019-11-21 17:06:47 +01:00 |
|
Adrián Chaves
|
a2bf340bab
|
Remove unused imports
|
2019-11-21 14:21:17 +01:00 |
|
Andrey Rakhmatullin
|
92ffd2f249
|
Simplify some more imports.
|
2019-11-14 13:45:39 +05:00 |
|
Ammar Najjar
|
da8cd9448d
|
docs: always surround __init__ with `` in docs
Issue #4086
|
2019-10-21 19:56:58 +02:00 |
|
Ammar Najjar
|
68a7d05ed8
|
docs: use __init__ method instead of constructor
Issue #4086
|
2019-10-21 15:42:24 +02:00 |
|
Eugenio Lacuesta
|
21ad8e20b9
|
Crawling rules: make link extractors optional
|
2019-09-13 17:32:42 -03:00 |
|
Eugenio Lacuesta
|
09e27d2d2e
|
[Tests] Monkey patch unittest.TestCase to prevent DeprecationWarning(s)
|
2019-07-13 19:26:59 -03:00 |
|
nyov
|
7a398b7086
|
Deprecation removals for Scrapy 1.7
Removing deprecations of 2015 and prior (pre-1.1)
|
2019-07-06 13:41:56 +00:00 |
|
Eugenio Lacuesta
|
1b4385b7e3
|
Rule.process_request: move deprecation warnings and compiling code, update tests
|
2019-03-22 19:46:17 -03:00 |
|
Eugenio Lacuesta
|
01ed605d02
|
PEP8 changes to test_spider.py
|
2019-03-15 16:54:14 +00:00 |
|
Eugenio Lacuesta
|
22fda61d62
|
Rule.process_request: tests
|
2019-03-13 10:54:38 +00:00 |
|
Victor Torres
|
fe283bcd05
|
add test case for sitemap filter with alternate links
|
2018-12-27 11:40:14 -03:00 |
|
Victor Torres
|
5e7ecf9dc1
|
add tests for sitemapindex
|
2018-12-27 11:40:14 -03:00 |
|
Victor Torres
|
657f0663b3
|
rename param from urls to entries
|
2018-12-27 11:40:14 -03:00 |
|
Victor Torres
|
d7d5917ff1
|
add tests for the sitemap_filter method in the SitemapSpider class
|
2018-12-27 11:40:14 -03:00 |
|
Mikhail Korobov
|
53da56c8dc
|
TST update tests to use get/getall/attrib instead of extract
|
2018-09-12 17:57:27 +05:00 |
|
Mikhail Korobov
|
885289f497
|
Merge pull request #2854 from jenya/2853-sitemap-follow-alternate-fix
[MRG+1] Follow alternate link for all types of sitemaps #2853
|
2017-08-21 19:15:03 +05:00 |
|
Daniel Graña
|
d5f3543d77
|
Merge pull request #2865 from kirankoduru/2831-explicit-msg-for-scrapy-parse-callback
[MRG+1] Explicit message for scrapy parse callback
|
2017-08-09 07:56:57 -03:00 |
|
Chomba Ng'ang'a
|
4ca61a2051
|
Update deprecated test aliases
- change ``failIf`` to ``assertFalse``
- change ``asertEquals`` to ``assertEqual``
- change ``assert_`` to ``assertTrue``
https://docs.python.org/2/library/unittest.html#deprecated-aliases
|
2017-08-09 12:52:10 +02:00 |
|
Kiran Koduru
|
12409a0cf6
|
Fix broken encoding on text for py 3
|
2017-08-05 15:40:38 -04:00 |
|
kirankoduru
|
7adab61a7a
|
Added test for NotImplemented Spider.parse method
|
2017-08-05 15:40:38 -04:00 |
|
Paul Tremberth
|
0cb3085f84
|
Add test for alternate links
|
2017-08-03 16:55:09 +02:00 |
|
Paul Tremberth
|
11cdf58abe
|
Always decompress Content-Encoding: gzip at HttpCompression stage
Let SitemapSpider handle decoding of .xml.gz files if necessary
|
2017-03-07 11:02:46 +01:00 |
|
Mikhail Korobov
|
a1e8a8525d
|
fix make_requests_from_url deprcation implementation, add tests
|
2017-02-17 00:18:29 +05:00 |
|
Eugenio Lacuesta
|
5ff64ad015
|
handle relative sitemap urls in robots.txt
|
2016-12-01 09:53:40 -03:00 |
|
orangain
|
25c56159b8
|
Fix SitemapSpider to extract sitemap urls from robots.txt properly
This will fix #1766.
|
2016-02-06 23:54:07 +09:00 |
|
Mikhail Korobov
|
d5984bbea9
|
PY3 port scrapy.spiders
|
2015-08-28 04:11:30 +05:00 |
|
Julia Medina
|
d3f576a816
|
Move scrapy/spider.py to scrapy/spiders/__init__.py
|
2015-05-09 04:20:09 -03:00 |
|
Julia Medina
|
fc346cba4d
|
Move scrapy/contrib/spiders to scrapy/spiders
|
2015-04-29 21:27:19 -03:00 |
|
Julia Medina
|
cf064b1437
|
Move scrapy/contrib/linkextractors to scrapy/linkextractors
|
2015-04-29 21:24:30 -03:00 |
|
Julia Medina
|
1c8708eb82
|
Create a logger for every Spider and adapt Spider.log to log through it
|
2015-04-22 17:24:40 -03:00 |
|
Berker Peksag
|
31e5f164d4
|
Import unittest.mock if available.
mock is in the stdlib since Python 3.3.
|
2015-03-08 15:03:04 +02:00 |
|
Julia Medina
|
9ef3972cfb
|
Per-spider settings tests
|
2014-09-01 21:56:57 -03:00 |
|
Julia Medina
|
870438e5f4
|
Update tests utils, fixing get_crawler and removing docrawl
|
2014-08-12 14:02:56 -03:00 |
|
Julia Medina
|
a995727117
|
Connect spider_closed signal after a crawler is bound to a Spider
|
2014-08-12 14:02:55 -03:00 |
|
Julia Medina
|
eb0253e530
|
Update from_crawler method as well as set_crawler on CrawlSpider
|
2014-08-11 11:24:01 -03:00 |
|
Julia Medina
|
84fa004793
|
Add from_crawler class method to base Spider
|
2014-08-11 11:23:57 -03:00 |
|
Daniel Graña
|
242c08553f
|
move tests under root dir
|
2014-07-30 19:45:08 -03:00 |
|