BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-23 12:04:00 +00:00
Code Releases Wiki Activity
4,472 Commits 39 Branches 119 Tags
Commit Graph

4472 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Pablo Hoffman
5846d6154c emphasize web crawling over screen scraping on scrapy description. closes #586 2015-03-18 00:24:16 -03:00
Mikhail Korobov
da90449edf typo fix in scrapy.contrib.pipeline.media 2015-03-18 07:24:15 +05:00
Mikhail Korobov
d14c972ed9 Merge pull request #1079 from kmike/feed-iterate-spider-output 
CSVFeedSpider cleanup: use iterate_spider_output
 2015-03-18 04:44:50 +05:00
Pablo Hoffman
3f15b6df8b Merge pull request #1078 from kmike/iter-spider-output 
remove unnecessary check from scrapy.utils.spider.iter_spider_output
 2015-03-17 16:36:43 -03:00
Mikhail Korobov
6974902323 CSVFeedSpider cleanup: use iterate_spider_output 
A similar fix was made for XMLFeedSpider in 95fde0a498
 2015-03-17 23:31:29 +05:00
Mikhail Korobov
8ce4ad0615 remove unnecessary check from scrapy.utils.spider.iter_spider_output 
arg_to_iter handles Items since 2bbd92742b
 2015-03-17 23:07:39 +05:00
Mikhail Korobov
62c4481d78 Merge pull request #992 from chekunkov/pydispatch_pep8 
Pydispatch pep8
 2015-03-17 22:42:31 +05:00
Pablo Hoffman
934584a355 Merge pull request #1020 from jojje/gzip_http_cache 
[MRG+1] add gzip compression to filesystem http cache backend
 2015-03-17 14:32:06 -03:00
Pablo Hoffman
f924567591 Merge pull request #983 from ArturGaspar/linkextractor_css 
[MRG+1] CSS support in link extractors
 2015-03-17 01:07:47 -03:00
Pablo Hoffman
b461c6f16f Merge pull request #821 from nramirezuy/httpcache_dont_cache-19-689 
[MRG+1] httpcache dont_cache meta #19 #689
 2015-03-17 01:04:36 -03:00
Daniel Graña
f52b1de69c Merge pull request #1077 from kmike/contribute-to-master 
DOC contribute to master branch
 2015-03-16 23:55:15 -03:00
Alexander Chekunkov
05cb31d306 pydispatch - pep8 
reformat files - proper spacing
 2015-03-16 21:43:43 +00:00
Mikhail Korobov
64082b4640 DOC contribute to master branch 
See GH-975 and GH-1029.
 2015-03-17 01:46:45 +05:00
nramirezuy
c13e23641b httpcache dont_cache meta #19 #689 2015-03-16 11:50:04 -03:00
Mikhail Korobov
c1bf8365d5 Merge pull request #1022 from scrapy/docs-reorder 
DOC reorder topics
 2015-03-13 16:40:47 +05:00
Mikhail Korobov
baf5c59386 Merge pull request #1071 from eliasdorneles/updating-request-meta-special-keys 
updating list of Request.meta special keys
 2015-03-13 16:38:19 +05:00
Elias Dorneles
57a5ee0097 added example value to set for proxy meta key 2015-03-12 23:20:44 -03:00
Elias Dorneles
f7031c08ff updating list of Request.meta special keys 2015-03-10 22:29:07 -03:00
Mikhail Korobov
645366a0fd Merge pull request #1070 from berkerpeksag/items-equal 
assertItemsEqual was renamed to assertCountEqual in Python 3.
 2015-03-08 22:54:57 +05:00
Mikhail Korobov
671d0a7523 Merge pull request #1066 from berkerpeksag/mock-import 
Import unittest.mock if available.
 2015-03-08 22:07:22 +05:00
Berker Peksag
c86e1bebec assertItemsEqual was renamed to assertCountEqual in Python 3. 2015-03-08 15:18:49 +02:00
Berker Peksag
31e5f164d4 Import unittest.mock if available. 
mock is in the stdlib since Python 3.3.
 2015-03-08 15:03:04 +02:00
Daniel Graña
c9d7386a32 Add 0.24.5 release notes 2015-02-25 19:36:40 -02:00
Julia Medina
ee17902923 Merge pull request #1049 from scrapy/twisted-15-support 
Twisted 15.0.0 Support
 2015-02-16 16:16:42 -03:00
Julia Medina
d67ca77e61 Support new _getEndpoint Agent signatures on Twisted 15.0.0 2015-02-15 21:14:47 -03:00
Mikhail Korobov
7a745b105c Merge pull request #1044 from klangner/master 
py3-ignores.txt supports comments
 2015-02-10 15:03:34 +05:00
klangner@gmail.com
a0299d97f5 fixed variable name 2015-02-10 10:34:44 +01:00
klangner
5ea4a72b8e py3-ignores.txt supports comments 2015-02-09 10:28:22 +01:00
Mikhail Korobov
75f1560ff5 Merge pull request #1033 from maikroeder/patch-1 
DOC typo fix in form.py
 2015-02-04 15:43:00 +05:00
Maik Röder
721d8d5dae Update form.py 
Typo fixed
 2015-02-04 11:38:37 +01:00
Mikhail Korobov
6419f76bbc Merge pull request #1029 from dangra/contribute-to-master 
DOC contribute to master branch
 2015-02-03 02:10:47 +05:00
Nicolás Alejandro Ramírez Quiros
8a3b9b6131 Merge pull request #1011 from SudShekhar/master 
Extension example fix to something that makes more sense
 2015-01-30 15:45:52 -02:00
Sudhanshu Shekhar
e42a1ac1a1 Reset items_scraped instead of item_count 
items_scraped is the counter that needs to be reset each time we have scraped a specific number of items in the code instead of item_count (which represents the specific number of items needed before a message is logged). Updating the source code to reflect this.
Removed some irrelevant words from the log message.
Signed-off-by: Sudhanshu Shekhar <sudshekhar02@gmail.com>
 2015-01-30 23:13:06 +05:30
Daniel Graña
f0bdc14522 Tentative attention message about what document to read for contributions 2015-01-29 15:40:12 -02:00
Daniel Graña
074b4a9315 Contribute to master branch 
ref #975
 2015-01-29 14:53:42 -02:00
Mikhail Korobov
a07b4353d6 DOC reorder topics 
* duplicate "topics/commands" link is removed;
* Request/Response docs are moved to "Basic Concepts";
* settings docs are moved to "Basic Concepts";
* exceptions docs are moved to "Basic Concepts";
* "signals" and "exporters" docs are moved to "Extending Scrapy";
* "Reference" section is dropped because it is empty now.
 2015-01-22 22:58:10 +05:00
Jonas Tingeborn
bd5d99a2d2 add gzip compression to filesystem http cache backend 2015-01-21 20:18:11 +01:00
Nicolás Alejandro Ramírez Quiros
2183984832 Merge pull request #1018 from barraponto/dynamic_item_practice_docs 
Updates documentation on dynamic item classes.
 2015-01-19 18:12:02 -02:00
Capi Etheriel
4bc14da59e Updates documentation on dynamic item classes. 
Fixes #398
 2015-01-19 17:21:56 -02:00
Mikhail Korobov
283d6a5344 DOC a couple more references are fixed 2015-01-19 22:07:03 +05:00
Mikhail Korobov
73e6b35622 DOC fix a reference 2015-01-19 22:02:46 +05:00
Daniel Graña
5bcb652410 Merge pull request #1017 from Curita/external-url-fix 
Don't rely on external urls for tests
 2015-01-19 13:57:18 -02:00
Julia Medina
d68615a5af Test the parse command locally instead of against an external url 2015-01-19 10:28:25 -03:00
Daniel Graña
f3110aaed1 Merge pull request #999 from Curita/fix-985 
Patch hanging HTTPConnectionPool.closeCachedConnections call
 2014-12-31 01:52:44 -02:00
Julia Medina
09ba4ff68a Patches Twisted issue while closing the connection pool on HTTPDownloadHandler 2014-12-30 19:53:50 -03:00
Nicolás Alejandro Ramírez Quiros
79c3274884 Merge pull request #987 from nyov/nyov/resolver 
t.i.b.ThreadedResolver is now a new-style class
 2014-12-17 11:28:59 -02:00
nyov
880c9e52b5 t.i.b.ThreadedResolver is now a new-style class 
since 2009 / twisted-9.0.0
663d669dce
 2014-12-16 19:28:39 +00:00
Pablo Hoffman
78954bc7f4 Merge pull request #978 from immerrr/fix-s3-authorization-for-quoted-paths 
S3DownloadHandler: fix auth for requests with quoted paths/query params
 2014-12-15 17:15:11 -02:00
Artur Gaspar
22247cf791 move restrict_css argument to end of argument list in link extractors for backwards compatibility, use keyword arguments in link extractor super().__init__() calls 2014-12-15 09:18:15 -02:00
Artur Gaspar
b0730a1d16 documentation for CSS support in link extractors 2014-12-11 18:22:08 -02:00