BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-28 13:24:31 +00:00
Code Releases Wiki Activity
5,153 Commits 39 Branches 119 Tags
Commit Graph

5153 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Leonid Amirov
bc9db65358 issue GH #1550 - scrapy shell argument fixes: "example.com" requests "http://example.com"; "example" requests "file://example"; "./example.com" requests "file://example.com" 2015-11-02 16:08:19 +03:00
Leonid Amirov
a41c64bfb9 issue GH #1550 - fixed bugs in scrapy.utils.url.add_http_if_no_scheme(): when given URI where scheme is present, but not 'http' the function gave bad result 2015-11-02 16:06:21 +03:00
Leonid Amirov
98a2e77a75 issue GH #1550 - fixed error: shell command wasn't accepting files URIs 2015-11-02 15:30:49 +03:00
Julia Medina
57f87b95d4 Merge pull request #1528 from scrapy/create-crawler 
public Crawler.create_crawler method
 2015-10-30 12:36:35 -03:00
Julia Medina
9424ca0fdb Merge pull request #1570 from jdemaeyer/tests/complete-settings-tests 
[MRG+1] Add some missing tests for scrapy.settings
 2015-10-30 12:36:25 -03:00
Mikhail Korobov
0000b6e9ef TST cleanup scrapy.utils.test.get_crawler 2015-10-30 20:27:03 +05:00
Mikhail Korobov
a49c82ad62 TST improve CrawlerRunner tests 
* use CrawlerRunner.create_crawler instead of get_crawler helper in test_crawl;
* add a test for loading spiders by name;
* add a test for passing Crawler objects instead of Spider objects;
* add a test for CrawlerRunner.join
 2015-10-30 20:24:37 +05:00
Mikhail Korobov
11b11c9803 CrawlerRunner.create_crawler method 2015-10-30 20:22:31 +05:00
Julia Medina
72eeead6db Merge pull request #1524 from Digenis/relocate_telnet_console 
[MRG+1] Relocate telnetconsole to extensions/
 2015-10-30 12:10:38 -03:00
Jakob de Maeyer
32ff4cc1d7 PEP8ify scrapy.settings tests 2015-10-29 19:52:19 +01:00
Jakob de Maeyer
51ca84c9b4 Add missing tests for scrapy.settings module 2015-10-29 17:34:18 +01:00
Mikhail Korobov
caf2080b80 Merge pull request #1423 from jdemaeyer/enhancement/executionengine-close 
[MRG+1] Add ExecutionEngine.close() method
 2015-10-29 19:11:07 +05:00
Jakob de Maeyer
8307c1212f Add ExecutionEngine.close() method 2015-10-29 10:03:29 +01:00
Daniel Graña
dd9f777ba7 Merge pull request #1149 from jdemaeyer/enhancement/settings-per-key-priorities 
[MRG+1] Per-key priorities for dict-like settings by promoting dicts to Settings instances
 2015-10-28 18:21:26 -03:00
Jakob de Maeyer
03f1720afb Fix backwards-compatibility for users who explicitly set _BASE settings 2015-10-27 14:22:56 +01:00
Jakob de Maeyer
f249b309ab Move scrapy.utils.conf.remove_none_values to s.u.python.without_none_values 2015-10-27 12:43:33 +01:00
Jakob de Maeyer
90198e5324 Add __repr__ method for BaseSettings 2015-10-27 12:43:33 +01:00
Jakob de Maeyer
9eb3597d15 PEP8ify settings module 2015-10-27 12:43:33 +01:00
Jakob de Maeyer
9bd7af8a62 Remove unused import in scrapy.settings 2015-10-27 12:43:33 +01:00
Jakob de Maeyer
bb6dee611c Move Settings documentation to docstrings 2015-10-27 12:43:33 +01:00
Jakob de Maeyer
26586ef5a6 Deprecate _BASE settings, unify _BASE backwards-compatibility 2015-10-27 12:43:23 +01:00
Jakob de Maeyer
a769a1ef78 Introduce BaseSettings with full dictionary interface 2015-10-27 12:38:52 +01:00
Mikhail Korobov
4a9f76ebd3 Merge pull request #1555 from stummjr/patch-1 
fix typo in the documentation
 2015-10-26 12:30:46 +05:00
Valdir Stumm Jr
d577c4702d fixed a typo in the documentation. 2015-10-26 00:00:20 -02:00
Pablo Hoffman
8dc400c66d Merge pull request #1541 from chripede/patch-1 
[MRG+1] Version 1 now exists
 2015-10-14 10:58:54 -03:00
Christian Pedersen
cc76fb3dff Version 1 now exists 2015-10-14 14:07:34 +02:00
Pablo Hoffman
451318ef7a Merge pull request #1530 from scrapy/retry-400 
[MRG+1] DOC fix docs after GH-1289.
 2015-10-13 16:09:07 -03:00
Daniel Graña
411174cf38 Merge pull request #1535 from scrapy/py3-commands 
PY3 port bench, startproject, genspider, list and runspider commands
 2015-10-12 20:43:21 -03:00
Daniel Graña
65f4ba349c Merge pull request #1537 from scrapy/no-coveralls 
drop coveralls support
 2015-10-12 20:36:12 -03:00
Mikhail Korobov
09fc9b48b6 drop coveralls support 2015-10-12 18:12:56 +05:00
Mikhail Korobov
215905bdb6 PY3 port bench, startproject, genspider, list and runspider commands 2015-10-12 17:59:49 +05:00
Julia Medina
0117c811c9 Merge pull request #1533 from Digenis/xml_iter-nodename_with_dot 
[MRG+1] xmliter nodename with dot
 2015-10-12 09:33:27 -03:00
Mikhail Korobov
1b6d60c251 DOC fix docs after GH-1289. 2015-10-09 01:26:09 +05:00
Mikhail Korobov
9676cf8e53 Merge pull request #1527 from scrapy/no-pillow-300 
TST don't use broken Pillow version in tests
 2015-10-09 01:20:45 +05:00
Νικόλαος-Διγενής Καραγιάννης
f56062d045 escape nodename in xmliter regex 2015-10-07 14:58:47 +03:00
Νικόλαος-Διγενής Καραγιάννης
d66efb13ba test xml nodename with dots 2015-10-07 14:43:47 +03:00
Mikhail Korobov
4e9a43c228 Merge pull request #1532 from hoatle/docs-fixes 
Docs fixes
 2015-10-07 15:34:21 +05:00
hoatle
2869cf8dde fix another invalid xpath error 2015-10-07 16:03:43 +07:00
Hoat Le
4e66955411 fix ValueError: Invalid XPath: //div/[id="not-exists"]/text() on selectors.rst 
>>> response.xpath('//div/[id="not-exists"]/text()').extract_first() is None
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/vagrant/.virtualenvs/scrapy/lib/python2.7/site-packages/scrapy/http/response/text.py", line 109, in xpath
    return self.selector.xpath(query)
  File "/home/vagrant/.virtualenvs/scrapy/lib/python2.7/site-packages/scrapy/selector/unified.py", line 100, in xpath
    raise ValueError(msg if six.PY3 else msg.encode("unicode_escape"))
ValueError: Invalid XPath: //div/[id="not-exists"]/text()
 2015-10-07 15:43:02 +07:00
Mikhail Korobov
669be0a2b0 TST don't use broken Pillow version in tests 2015-10-06 17:24:52 +05:00
Νικόλαος-Διγενής Καραγιάννης
d523c75f5e Relocate telnetconsole to extensions/ 2015-10-05 21:16:04 +03:00
Mikhail Korobov
91cbf97415 Merge pull request #1522 from smirecki/fix-typos-in-docs-topics 
DOC Typos corrections
 2015-10-03 13:23:38 +05:00
smirecki
8379bea7ed Typos corrections 
I've made a few small corrections, some spelling changes and typo fixes.
I've tried to respect regional spelling differences and avoided proposing hyphenating compound words.

 Please enter the commit message for your changes. Lines starting
 2015-10-02 23:48:27 -04:00
Daniel Graña
fe15f93e53 Merge pull request #1498 from Preetwinder/scrapy-add_scheme 
[MRG+1] #1487 add_scheme_if_missing for `scrapy shell` command
 2015-09-30 21:38:58 -03:00
Marius Gedminas
0620e76433 Fix list formatting 2015-09-29 03:33:30 +05:00
Marius Gedminas
eaad10facf Typo 2015-09-28 14:44:15 +05:00
Pablo Hoffman
75cd056223 disable log on version command. closes #1426 2015-09-25 16:39:21 -03:00
preetwinder
47c8e2ba78 Restructure tests for add_http_if_no_scheme function 2015-09-24 17:57:25 +00:00
preetwinder
9d96e767a1 Minor changes to add_http_if_no_scheme 2015-09-24 17:31:30 +00:00
preetwinder
8c629eee3e adds docstring, tests and correction 2015-09-24 17:28:44 +00:00