BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-28 16:36:32 +00:00
Code Releases Wiki Activity
5,387 Commits 39 Branches 119 Tags
Commit Graph

411 Commits

Author SHA1 Message Date
nramirezuy
7871acd665 Item multi inherit fixed 2015-04-21 18:10:52 -03:00
Mikhail Korobov
c3d3a94914 fixed backwards compatibility for SPIDER_MANAGER_CLASS option 2015-04-22 00:29:40 +05:00
Pablo Hoffman
0a5bbbaed3 Merge pull request #1159 from scrapy/feed-export-fields 
[MRG+1] FEED_EXPORT_FIELDS option
 2015-04-21 15:54:24 -03:00
Daniel Graña
e034947b54 Merge pull request #1166 from scrapy/spider-loader 
[MRG+1] rename SpiderManager to SpiderLoader
 2015-04-21 15:42:17 -03:00
Pablo Hoffman
06e1ca9ef2 Merge pull request #1177 from Curita/remove-djangoitem 
Remove djangoitem
 2015-04-21 15:35:47 -03:00
Nikolaos-Digenis Karagiannis
017fb25bee loader test with processors that use item's values 2015-04-20 11:04:31 +03:00
Julia Medina
ffc60910aa Remove djangoitem since we moved it to scrapy/scrapy-djangoitem 2015-04-19 13:09:25 -03:00
Mikhail Korobov
ad587ea792 rename CrawlerRunner.spiders to CrawlerRunner.spider_loader 2015-04-17 01:54:26 +05:00
Mikhail Korobov
403e7c7c70 rename scrapy.spidermanager.SpiderManager to scrapy.spiderloader.SpiderLoader 2015-04-16 20:07:53 +05:00
mrpandav
954c8fcecb changes test_from_response_formname_notexists_fallback_formid unit test for pull request #1137 - addition of new shortcut for html form election by formid attribute 2015-04-15 16:49:09 +05:30
mrpandav
020a32a3d9 Adding tests for pull request #1137 - addition of new shortcut for html form election by formid attribute 2015-04-15 11:23:25 +05:30
Julia Medina
526aa07f44 Merge pull request #1156 from kmike/crawlers-accept-dicts 
allow Crawler, CrawlerRunner and CrawlerProcess to accept dicts instead of Setting objects
 2015-04-15 02:15:32 -03:00
Daniel Graña
c013baa602 Merge pull request #1161 from scrapy/telnet-disable 
disable scrapy.telnet if twisted.conch is not available
 2015-04-15 00:32:42 -03:00
Mikhail Korobov
0b76487629 PY3 fix Python 3 syntax errors in ajaxcrawl.py 
Tests for _has_ajaxcrawlable_meta passin Python 3.
 2015-04-15 04:16:26 +05:00
Mikhail Korobov
378b6efc0e PY3 disable scrapy.telnet if twisted.conch is not available 2015-04-15 04:06:36 +05:00
Mikhail Korobov
abeb8e3a01 TST skip a test in Python 3 because it imports some parts of Twisted which are not ported yet 2015-04-15 03:27:11 +05:00
Mikhail Korobov
647eeaea61 TST add a test for GH-1050. 2015-04-15 03:23:30 +05:00
Mikhail Korobov
1534e8540b FEED_EXPORT_FIELDS option 2015-04-15 02:48:26 +05:00
Mikhail Korobov
7644813286 allow Crawler, CrawlerRunner and CrawlerProcess to accept dicts instead of Setting objects 2015-04-14 23:26:05 +05:00
Mikhail Korobov
973c31f735 TST cleanup: use assertIn instead of assert_ 2015-04-14 20:11:46 +05:00
Pablo Hoffman
71c0aface7 Merge pull request #1148 from Curita/verify-spidermanager-interface 
Verify SPIDER_MANAGER_CLASS interface while loading it in CrawlerRunner
 2015-04-13 14:25:32 -03:00
Jakob de Maeyer
cf9d848f0f Fixed deprecated settings API in cmdline test 2015-04-12 14:18:47 +02:00
Nikolaos-Digenis Karagiannis
bf301b7328 encode invalid xpath with unicode_escape under PY2 
The exception quotes an xpath string
which may be unicode.
 2015-04-10 22:23:40 +03:00
Julia Medina
24a07fd892 Verify SPIDER_MANAGER_CLASS interface in CrawlerRunner 2015-04-10 13:43:33 -03:00
nyov
6d48c19acd dissolve scrapy.contrib_exp 2015-04-07 21:20:48 +00:00
Pablo Hoffman
4b11501eb5 Merge pull request #963 from tpeng/fix-xmliter-lxml 
[MRG+1] support namespace prefix in xmliter_lxml
 2015-04-07 18:13:01 -03:00
Daniel Graña
9706119336 Merge pull request #1128 from Curita/move-spider-settings-population-to-crawler 
[MRG+1] Move spider settings population from CrawlerRunner to Crawler.__init__
 2015-04-04 18:08:05 -03:00
Julia Medina
6f9265b44b Move spider settings population to Crawler.__init__ 2015-04-01 23:29:10 -03:00
nyov
dc88be75e2 more python3 compatibility changes for urllib 2015-04-01 19:50:56 +00:00
Daniel Graña
ec4251afd7 Merge pull request #1089 from drack3800/master 
[MRG+1] Add test for webclient with POST method and no body given
 2015-03-31 17:30:08 -03:00
Pablo Hoffman
bb4c922d85 Merge pull request #1081 from scrapy/dict-items 
Allow spiders to return dicts.
 2015-03-27 15:19:27 -03:00
nyov
84edc2ebc7 Add Response.urljoin() testcase 
and add evaluation of base-url for HtmlResponse.
 2015-03-27 05:59:01 +00:00
drack3800
1b6d5a011a Added webclient test for checking Content-Length header in response for POST request with no given body 2015-03-21 04:02:51 +03:00
Mateusz Golewski
f92bc09bf4 Add re_first() to SelectorList and iflatten() to utils.python 2015-03-18 21:11:08 -03:00
Mateusz Golewski
2742b4d8c2 Add tests to extract_first() 2015-03-18 21:11:08 -03:00
Nicolás Alejandro Ramírez Quiros
ee82fe0e24 Merge pull request #1016 from SudShekhar/jsonProcessor 
[MRG+1] Added JmesSelect
 2015-03-18 08:11:25 -03:00
Mikhail Korobov
39635e5f55 Allow spiders to return dicts. See GH-1064. 2015-03-18 07:26:56 +05:00
Pablo Hoffman
934584a355 Merge pull request #1020 from jojje/gzip_http_cache 
[MRG+1] add gzip compression to filesystem http cache backend
 2015-03-17 14:32:06 -03:00
Pablo Hoffman
f924567591 Merge pull request #983 from ArturGaspar/linkextractor_css 
[MRG+1] CSS support in link extractors
 2015-03-17 01:07:47 -03:00
nramirezuy
c13e23641b httpcache dont_cache meta #19 #689 2015-03-16 11:50:04 -03:00
Mikhail Korobov
645366a0fd Merge pull request #1070 from berkerpeksag/items-equal 
assertItemsEqual was renamed to assertCountEqual in Python 3.
 2015-03-08 22:54:57 +05:00
Berker Peksag
c86e1bebec assertItemsEqual was renamed to assertCountEqual in Python 3. 2015-03-08 15:18:49 +02:00
Berker Peksag
31e5f164d4 Import unittest.mock if available. 
mock is in the stdlib since Python 3.3.
 2015-03-08 15:03:04 +02:00
Sudhanshu Shekhar
839ffba971 Added the first version of SelectJmes 
Utilizes jmespath. Also, added tests and documentation for the same.
 2015-02-24 22:59:01 +05:30
Jonas Tingeborn
bd5d99a2d2 add gzip compression to filesystem http cache backend 2015-01-21 20:18:11 +01:00
Julia Medina
d68615a5af Test the parse command locally instead of against an external url 2015-01-19 10:28:25 -03:00
tpeng
82d138e87e support namespace prefix in xmliter_lxml 2014-12-15 16:17:06 +01:00
Artur Gaspar
403fc686b8 tests for CSS support in link extractors 2014-12-11 18:20:30 -02:00
immerrr
82b187f283 S3DownloadHandler: fix auth for requests with quoted paths/query params 2014-12-11 18:14:36 +03:00
Lev Berman
fdb6bb07c0 #977 - test dropping requests 2014-11-28 10:53:33 +03:00