BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-28 09:23:51 +00:00
Code Releases Wiki Activity
4,711 Commits 39 Branches 119 Tags
Commit Graph

4711 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
drack3800
6afd1a78bd style fix 2015-03-25 20:43:07 +03:00
Daniel Graña
ad36de4e62 Merge pull request #1101 from nyov/nyov/tls-sni 
handle TLS SNI
 2015-03-24 18:21:10 -03:00
nyov
aaeb837db4 handle TLS SNI if we have twisted>=14.0 
(closes #981, #1101)
 2015-03-24 10:52:29 +00:00
Mikhail Korobov
5ac91e4883 DOC remove Dynamic Creation of Item Classes section 
It was a hack, and dicts-as-items cover most use cases.

Dicts don't allow to attach metadata to fields,
but e.g. adding "_meta" key and removing it in a custom serializer
is no worse than creating classes dynamically.
 2015-03-23 18:11:35 +05:00
drack3800
549882590b Fixed bug with no specified Content-Length header by ScrapyHTTPClientFactory for POST request with no given body 2015-03-22 19:31:29 +03:00
drack3800
deb5bb530c Fixed bug with no specified Content-Length header by ScrapyHTTPClientFactory for POST request with no given body 2015-03-22 19:25:08 +03:00
drack3800
1b6d5a011a Added webclient test for checking Content-Length header in response for POST request with no given body 2015-03-21 04:02:51 +03:00
Julia Medina
cda3922507 Add Response.urljoin() helper 2015-03-19 19:07:52 -03:00
Pablo Hoffman
c81eefaf81 fix doc links 2015-03-19 17:42:48 -03:00
Elias Dorneles
d7cb2b9a91 making commented code indentation consistent 2015-03-19 15:41:43 -03:00
Mikhail Korobov
8ac397670f DOC move .. module: declaration to a proper place 2015-03-19 21:41:36 +05:00
Julia Medina
67a85c77f2 Merge branch 'coder46-master' from pull request #647 2015-03-18 23:55:42 -03:00
Faisal Anees
643984e1b4 Updated architecture.rst 
Added http://krondo.com/blog/?page_id=1327 as a resource
 2015-03-18 23:55:22 -03:00
Julia Medina
ff64584876 Merge branch 'ananana-selectorlist-extract-first' from pull request #624 2015-03-18 21:29:49 -03:00
Mikhail Korobov
f16a33f34e DOC change structure of spider docs: 
* start with scrapy.Spider, then mention spider arguments,
  then describe generic spiders;
* change wording regarding start_urls/start_requests;
* show an example of start_requests vs start_urls;
* show an example of dicts as items;
* as defining Item is an optional step now, docs for Items are
  moved below Spider docs.
 2015-03-19 05:25:15 +05:00
Mikhail Korobov
817dbc6cbd DOC mention dicts in documentation; explain better what are Items for 2015-03-19 05:16:14 +05:00
Julia Medina
959aaad205 Document re_first 2015-03-18 21:11:08 -03:00
Julia Medina
0dade7315b Use generator sintax in re_first 2015-03-18 21:11:08 -03:00
Mateusz Golewski
f92bc09bf4 Add re_first() to SelectorList and iflatten() to utils.python 2015-03-18 21:11:08 -03:00
Mateusz Golewski
127c6c694a Fix extract_first() docs 2015-03-18 21:11:08 -03:00
Mateusz Golewski
012211accd Add docs for extract_first() 2015-03-18 21:11:08 -03:00
Mateusz Golewski
2742b4d8c2 Add tests to extract_first() 2015-03-18 21:11:08 -03:00
Mateusz Golewski
bd126be356 Optimize extract_first() 2015-03-18 21:11:08 -03:00
Mateusz Golewski
6c7bd54fc3 Add extract_first() method to SelectorList 2015-03-18 21:11:08 -03:00
Daniel Graña
12eedd90d1 fix truncated 0.24.5 release notes. closes #1084 2015-03-18 20:46:28 -03:00
Julia Medina
e17f97e637 Merge branch 'dufferzafar-fix-linkcheck-606' from pull request #1041 2015-03-18 20:05:50 -03:00
Julia Medina
4fb818a250 Run linkfix over current docs 2015-03-18 20:04:14 -03:00
Julia Medina
ff987fb5a5 Add linkfix rule to docs Makefile 2015-03-18 20:00:03 -03:00
Shadab Zafar
c05f5f175e Added linkfix script to docs/utils 
https://github.com/scrapy/scrapy/pull/1041#issuecomment-78143576
 2015-03-18 19:41:26 -03:00
Shadab Zafar
5a58d64131 Fix some redirection links in documentation 
Fixes #606
 2015-03-18 19:41:26 -03:00
Mikhail Korobov
4c11201de6 Merge pull request #1082 from iKevinY/readme-badges 
Use Shields.io SVGs for README badges
 2015-03-18 23:33:10 +05:00
Nicolás Alejandro Ramírez Quiros
ee82fe0e24 Merge pull request #1016 from SudShekhar/jsonProcessor 
[MRG+1] Added JmesSelect
 2015-03-18 08:11:25 -03:00
Kevin Yap
776616bdec Use Shields.io SVGs for README badges 
- Use SVGs for badges (more friendly to retina displays).
- Add alt text to PyPI version and build status badges.
 2015-03-17 22:26:06 -07:00
Pablo Hoffman
5846d6154c emphasize web crawling over screen scraping on scrapy description. closes #586 2015-03-18 00:24:16 -03:00
Mikhail Korobov
39635e5f55 Allow spiders to return dicts. See GH-1064. 2015-03-18 07:26:56 +05:00
Mikhail Korobov
da90449edf typo fix in scrapy.contrib.pipeline.media 2015-03-18 07:24:15 +05:00
Mikhail Korobov
d14c972ed9 Merge pull request #1079 from kmike/feed-iterate-spider-output 
CSVFeedSpider cleanup: use iterate_spider_output
 2015-03-18 04:44:50 +05:00
Pablo Hoffman
3f15b6df8b Merge pull request #1078 from kmike/iter-spider-output 
remove unnecessary check from scrapy.utils.spider.iter_spider_output
 2015-03-17 16:36:43 -03:00
Mikhail Korobov
6974902323 CSVFeedSpider cleanup: use iterate_spider_output 
A similar fix was made for XMLFeedSpider in 95fde0a498
 2015-03-17 23:31:29 +05:00
Mikhail Korobov
8ce4ad0615 remove unnecessary check from scrapy.utils.spider.iter_spider_output 
arg_to_iter handles Items since 2bbd92742b
 2015-03-17 23:07:39 +05:00
Mikhail Korobov
62c4481d78 Merge pull request #992 from chekunkov/pydispatch_pep8 
Pydispatch pep8
 2015-03-17 22:42:31 +05:00
Pablo Hoffman
934584a355 Merge pull request #1020 from jojje/gzip_http_cache 
[MRG+1] add gzip compression to filesystem http cache backend
 2015-03-17 14:32:06 -03:00
Elias Dorneles
fd67fe273a using default values for settings that are off by default 2015-03-17 09:34:54 -03:00
Pablo Hoffman
f924567591 Merge pull request #983 from ArturGaspar/linkextractor_css 
[MRG+1] CSS support in link extractors
 2015-03-17 01:07:47 -03:00
Pablo Hoffman
b461c6f16f Merge pull request #821 from nramirezuy/httpcache_dont_cache-19-689 
[MRG+1] httpcache dont_cache meta #19 #689
 2015-03-17 01:04:36 -03:00
Daniel Graña
f52b1de69c Merge pull request #1077 from kmike/contribute-to-master 
DOC contribute to master branch
 2015-03-16 23:55:15 -03:00
Alexander Chekunkov
05cb31d306 pydispatch - pep8 
reformat files - proper spacing
 2015-03-16 21:43:43 +00:00
Mikhail Korobov
64082b4640 DOC contribute to master branch 
See GH-975 and GH-1029.
 2015-03-17 01:46:45 +05:00
Elias Dorneles
aa56dd30ea updating new settings to template, as pointed in PR review 2015-03-16 17:36:46 -03:00
nramirezuy
c13e23641b httpcache dont_cache meta #19 #689 2015-03-16 11:50:04 -03:00