BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-27 06:43:46 +00:00
Code Releases Wiki Activity
4,698 Commits 39 Branches 119 Tags
Commit Graph

4698 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Julia Medina
ff64584876 Merge branch 'ananana-selectorlist-extract-first' from pull request #624 2015-03-18 21:29:49 -03:00
Mikhail Korobov
f16a33f34e DOC change structure of spider docs: 
* start with scrapy.Spider, then mention spider arguments,
  then describe generic spiders;
* change wording regarding start_urls/start_requests;
* show an example of start_requests vs start_urls;
* show an example of dicts as items;
* as defining Item is an optional step now, docs for Items are
  moved below Spider docs.
 2015-03-19 05:25:15 +05:00
Mikhail Korobov
817dbc6cbd DOC mention dicts in documentation; explain better what are Items for 2015-03-19 05:16:14 +05:00
Julia Medina
959aaad205 Document re_first 2015-03-18 21:11:08 -03:00
Julia Medina
0dade7315b Use generator sintax in re_first 2015-03-18 21:11:08 -03:00
Mateusz Golewski
f92bc09bf4 Add re_first() to SelectorList and iflatten() to utils.python 2015-03-18 21:11:08 -03:00
Mateusz Golewski
127c6c694a Fix extract_first() docs 2015-03-18 21:11:08 -03:00
Mateusz Golewski
012211accd Add docs for extract_first() 2015-03-18 21:11:08 -03:00
Mateusz Golewski
2742b4d8c2 Add tests to extract_first() 2015-03-18 21:11:08 -03:00
Mateusz Golewski
bd126be356 Optimize extract_first() 2015-03-18 21:11:08 -03:00
Mateusz Golewski
6c7bd54fc3 Add extract_first() method to SelectorList 2015-03-18 21:11:08 -03:00
Daniel Graña
12eedd90d1 fix truncated 0.24.5 release notes. closes #1084 2015-03-18 20:46:28 -03:00
Julia Medina
e17f97e637 Merge branch 'dufferzafar-fix-linkcheck-606' from pull request #1041 2015-03-18 20:05:50 -03:00
Julia Medina
4fb818a250 Run linkfix over current docs 2015-03-18 20:04:14 -03:00
Julia Medina
ff987fb5a5 Add linkfix rule to docs Makefile 2015-03-18 20:00:03 -03:00
Shadab Zafar
c05f5f175e Added linkfix script to docs/utils 
https://github.com/scrapy/scrapy/pull/1041#issuecomment-78143576
 2015-03-18 19:41:26 -03:00
Shadab Zafar
5a58d64131 Fix some redirection links in documentation 
Fixes #606
 2015-03-18 19:41:26 -03:00
Mikhail Korobov
4c11201de6 Merge pull request #1082 from iKevinY/readme-badges 
Use Shields.io SVGs for README badges
 2015-03-18 23:33:10 +05:00
Nicolás Alejandro Ramírez Quiros
ee82fe0e24 Merge pull request #1016 from SudShekhar/jsonProcessor 
[MRG+1] Added JmesSelect
 2015-03-18 08:11:25 -03:00
Kevin Yap
776616bdec Use Shields.io SVGs for README badges 
- Use SVGs for badges (more friendly to retina displays).
- Add alt text to PyPI version and build status badges.
 2015-03-17 22:26:06 -07:00
Pablo Hoffman
5846d6154c emphasize web crawling over screen scraping on scrapy description. closes #586 2015-03-18 00:24:16 -03:00
Mikhail Korobov
39635e5f55 Allow spiders to return dicts. See GH-1064. 2015-03-18 07:26:56 +05:00
Mikhail Korobov
da90449edf typo fix in scrapy.contrib.pipeline.media 2015-03-18 07:24:15 +05:00
Mikhail Korobov
d14c972ed9 Merge pull request #1079 from kmike/feed-iterate-spider-output 
CSVFeedSpider cleanup: use iterate_spider_output
 2015-03-18 04:44:50 +05:00
Pablo Hoffman
3f15b6df8b Merge pull request #1078 from kmike/iter-spider-output 
remove unnecessary check from scrapy.utils.spider.iter_spider_output
 2015-03-17 16:36:43 -03:00
Mikhail Korobov
6974902323 CSVFeedSpider cleanup: use iterate_spider_output 
A similar fix was made for XMLFeedSpider in 95fde0a498
 2015-03-17 23:31:29 +05:00
Mikhail Korobov
8ce4ad0615 remove unnecessary check from scrapy.utils.spider.iter_spider_output 
arg_to_iter handles Items since 2bbd92742b
 2015-03-17 23:07:39 +05:00
Mikhail Korobov
62c4481d78 Merge pull request #992 from chekunkov/pydispatch_pep8 
Pydispatch pep8
 2015-03-17 22:42:31 +05:00
Pablo Hoffman
934584a355 Merge pull request #1020 from jojje/gzip_http_cache 
[MRG+1] add gzip compression to filesystem http cache backend
 2015-03-17 14:32:06 -03:00
Elias Dorneles
fd67fe273a using default values for settings that are off by default 2015-03-17 09:34:54 -03:00
Pablo Hoffman
f924567591 Merge pull request #983 from ArturGaspar/linkextractor_css 
[MRG+1] CSS support in link extractors
 2015-03-17 01:07:47 -03:00
Pablo Hoffman
b461c6f16f Merge pull request #821 from nramirezuy/httpcache_dont_cache-19-689 
[MRG+1] httpcache dont_cache meta #19 #689
 2015-03-17 01:04:36 -03:00
Daniel Graña
f52b1de69c Merge pull request #1077 from kmike/contribute-to-master 
DOC contribute to master branch
 2015-03-16 23:55:15 -03:00
Alexander Chekunkov
05cb31d306 pydispatch - pep8 
reformat files - proper spacing
 2015-03-16 21:43:43 +00:00
Mikhail Korobov
64082b4640 DOC contribute to master branch 
See GH-975 and GH-1029.
 2015-03-17 01:46:45 +05:00
Elias Dorneles
aa56dd30ea updating new settings to template, as pointed in PR review 2015-03-16 17:36:46 -03:00
nramirezuy
c13e23641b httpcache dont_cache meta #19 #689 2015-03-16 11:50:04 -03:00
Elias Dorneles
05a88152af adding more settings to project template 2015-03-14 16:12:37 -03:00
Mikhail Korobov
c1bf8365d5 Merge pull request #1022 from scrapy/docs-reorder 
DOC reorder topics
 2015-03-13 16:40:47 +05:00
Mikhail Korobov
baf5c59386 Merge pull request #1071 from eliasdorneles/updating-request-meta-special-keys 
updating list of Request.meta special keys
 2015-03-13 16:38:19 +05:00
Elias Dorneles
57a5ee0097 added example value to set for proxy meta key 2015-03-12 23:20:44 -03:00
Elias Dorneles
f7031c08ff updating list of Request.meta special keys 2015-03-10 22:29:07 -03:00
Mikhail Korobov
645366a0fd Merge pull request #1070 from berkerpeksag/items-equal 
assertItemsEqual was renamed to assertCountEqual in Python 3.
 2015-03-08 22:54:57 +05:00
Mikhail Korobov
671d0a7523 Merge pull request #1066 from berkerpeksag/mock-import 
Import unittest.mock if available.
 2015-03-08 22:07:22 +05:00
Berker Peksag
c86e1bebec assertItemsEqual was renamed to assertCountEqual in Python 3. 2015-03-08 15:18:49 +02:00
Berker Peksag
31e5f164d4 Import unittest.mock if available. 
mock is in the stdlib since Python 3.3.
 2015-03-08 15:03:04 +02:00
Daniel Graña
c9d7386a32 Add 0.24.5 release notes 2015-02-25 19:36:40 -02:00
Sudhanshu Shekhar
839ffba971 Added the first version of SelectJmes 
Utilizes jmespath. Also, added tests and documentation for the same.
 2015-02-24 22:59:01 +05:30
Julia Medina
ee17902923 Merge pull request #1049 from scrapy/twisted-15-support 
Twisted 15.0.0 Support
 2015-02-16 16:16:42 -03:00
Julia Medina
d67ca77e61 Support new _getEndpoint Agent signatures on Twisted 15.0.0 2015-02-15 21:14:47 -03:00