1
0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-25 19:44:02 +00:00

4530 Commits

Author SHA1 Message Date
Elias Dorneles
fd1c688ad6 addressing issues from review at #1150 2015-04-15 14:21:48 -03:00
Elias Dorneles
dce48b8654 documenting Files Pipeline together with Images Pipeline 2015-04-11 13:57:55 -03:00
Elias Dorneles
0c74821805 Merge pull request #1112 from eliasdorneles/minor-grammar-fixes
Some minor grammar fixes
2015-03-27 21:23:55 -03:00
Elias Dorneles
18c5bc75c2 some more minor grammar fixes 2015-03-27 21:00:21 -03:00
Elias Dorneles
7135c4e694 some minor grammar fixes 2015-03-27 17:56:56 -03:00
Nicolás Alejandro Ramírez Quiros
a3702e749f Merge pull request #1073 from eliasdorneles/add-more-settings-to-template
[MRG+1] Adding more settings to project template
2015-03-27 15:37:26 -03:00
Pablo Hoffman
bb4c922d85 Merge pull request #1081 from scrapy/dict-items
Allow spiders to return dicts.
2015-03-27 15:19:27 -03:00
Daniel Graña
55a23d102f Merge pull request #1086 from Curita/response-urljoin
Add Response.urljoin() helper
2015-03-27 15:17:54 -03:00
Julia Medina
f4e241a018 Merge pull request #1106 from eliasdorneles/overview-page-improvements
[MRG+1] some improvements to overview page
2015-03-27 15:16:33 -03:00
nyov
84edc2ebc7 Add Response.urljoin() testcase
and add evaluation of base-url for HtmlResponse.
2015-03-27 05:59:01 +00:00
Mikhail Korobov
39085ae18f Merge pull request #1098 from nyov/nyov/userconfig
[+1 MRG]look in ~/.config/scrapy.cfg for user config
2015-03-27 02:11:35 +05:00
nyov
1134a9cab0 config: look in ~/.config/scrapy.cfg as well 2015-03-26 20:36:14 +00:00
Mikhail Korobov
1ce0425542 Merge pull request #1108 from pbronez/master
Converted sel.xpath() calls to response.xpath() in Extracting the data. Fixes GH-1107.
2015-03-27 00:52:17 +05:00
Peter Bronez
475766c73a Converted sel.xpath() calls to response.xpath() in Extracting the data 2015-03-26 15:34:30 -04:00
Elias Dorneles
4dcecc98f9 moved example data to a better place 2015-03-26 15:45:17 -03:00
Elias Dorneles
7402e27230 fix community link 2015-03-26 15:35:31 -03:00
Elias Dorneles
729861c864 fixing indentation 2015-03-26 15:31:42 -03:00
Elias Dorneles
13d0ecde77 addressing more review comments, to avoid ambiguity on desired reading flow 2015-03-26 15:26:16 -03:00
Elias Dorneles
76e3bf1250 addressing comments from the review plus further editing 2015-03-26 14:26:20 -03:00
Elias Dorneles
8f4a268f37 added bit about async requests, improved phrasing 2015-03-26 12:14:56 -03:00
Elias Dorneles
32423d4a33 some improvements to overview page 2015-03-25 19:27:52 -03:00
Mikhail Korobov
e3e60da6c2 Merge pull request #1105 from ramiro/patch-1
Add missing callback arg in jobs topic example.
2015-03-26 03:19:39 +05:00
Ramiro Morales
933dbc6be6 Oops 2015-03-25 18:33:17 -03:00
Ramiro Morales
ca2575001e Add missing callback arg in jobs topic example. 2015-03-25 18:32:20 -03:00
Daniel Graña
ad36de4e62 Merge pull request #1101 from nyov/nyov/tls-sni
handle TLS SNI
2015-03-24 18:21:10 -03:00
nyov
aaeb837db4 handle TLS SNI if we have twisted>=14.0
(closes #981, #1101)
2015-03-24 10:52:29 +00:00
Mikhail Korobov
5ac91e4883 DOC remove Dynamic Creation of Item Classes section
It was a hack, and dicts-as-items cover most use cases.

Dicts don't allow to attach metadata to fields,
but e.g. adding "_meta" key and removing it in a custom serializer
is no worse than creating classes dynamically.
2015-03-23 18:11:35 +05:00
Julia Medina
cda3922507 Add Response.urljoin() helper 2015-03-19 19:07:52 -03:00
Pablo Hoffman
c81eefaf81 fix doc links 2015-03-19 17:42:48 -03:00
Elias Dorneles
d7cb2b9a91 making commented code indentation consistent 2015-03-19 15:41:43 -03:00
Mikhail Korobov
8ac397670f DOC move .. module: declaration to a proper place 2015-03-19 21:41:36 +05:00
Julia Medina
67a85c77f2 Merge branch 'coder46-master' from pull request #647 2015-03-18 23:55:42 -03:00
Faisal Anees
643984e1b4 Updated architecture.rst
Added http://krondo.com/blog/?page_id=1327 as a resource
2015-03-18 23:55:22 -03:00
Julia Medina
ff64584876 Merge branch 'ananana-selectorlist-extract-first' from pull request #624 2015-03-18 21:29:49 -03:00
Mikhail Korobov
f16a33f34e DOC change structure of spider docs:
* start with scrapy.Spider, then mention spider arguments,
  then describe generic spiders;
* change wording regarding start_urls/start_requests;
* show an example of start_requests vs start_urls;
* show an example of dicts as items;
* as defining Item is an optional step now, docs for Items are
  moved below Spider docs.
2015-03-19 05:25:15 +05:00
Mikhail Korobov
817dbc6cbd DOC mention dicts in documentation; explain better what are Items for 2015-03-19 05:16:14 +05:00
Julia Medina
959aaad205 Document re_first 2015-03-18 21:11:08 -03:00
Julia Medina
0dade7315b Use generator sintax in re_first 2015-03-18 21:11:08 -03:00
Mateusz Golewski
f92bc09bf4 Add re_first() to SelectorList and iflatten() to utils.python 2015-03-18 21:11:08 -03:00
Mateusz Golewski
127c6c694a Fix extract_first() docs 2015-03-18 21:11:08 -03:00
Mateusz Golewski
012211accd Add docs for extract_first() 2015-03-18 21:11:08 -03:00
Mateusz Golewski
2742b4d8c2 Add tests to extract_first() 2015-03-18 21:11:08 -03:00
Mateusz Golewski
bd126be356 Optimize extract_first() 2015-03-18 21:11:08 -03:00
Mateusz Golewski
6c7bd54fc3 Add extract_first() method to SelectorList 2015-03-18 21:11:08 -03:00
Daniel Graña
12eedd90d1 fix truncated 0.24.5 release notes. closes #1084 2015-03-18 20:46:28 -03:00
Julia Medina
e17f97e637 Merge branch 'dufferzafar-fix-linkcheck-606' from pull request #1041 2015-03-18 20:05:50 -03:00
Julia Medina
4fb818a250 Run linkfix over current docs 2015-03-18 20:04:14 -03:00
Julia Medina
ff987fb5a5 Add linkfix rule to docs Makefile 2015-03-18 20:00:03 -03:00
Shadab Zafar
c05f5f175e Added linkfix script to docs/utils
https://github.com/scrapy/scrapy/pull/1041#issuecomment-78143576
2015-03-18 19:41:26 -03:00
Shadab Zafar
5a58d64131 Fix some redirection links in documentation
Fixes #606
2015-03-18 19:41:26 -03:00