Alexander Sibiryakov
85aa3c7596
Dns cache size and timeout options
2015-04-02 18:30:59 +02:00
Julia Medina
66aef0488a
Merge pull request #1127 from nyov/docs-fix
...
documentation build warning fixes
2015-04-01 17:02:28 -03:00
nyov
92b574309e
documentation build warning fixes
2015-04-01 19:46:21 +00:00
Daniel Graña
27591b55fc
Merge pull request #1123 from sibiryakov/reactor-threadpool-size
...
[MRG+1] Reactor threadpool max size setting
2015-04-01 15:08:03 -03:00
Alexander Sibiryakov
b794cdaf4b
Broad crawls notes.
2015-04-01 12:07:03 +02:00
Alexander Sibiryakov
e7b274edf3
Reformat to 80 characters per line.
2015-04-01 11:49:55 +02:00
Alexander Sibiryakov
94fceb4c15
Fixing underscore size.
2015-04-01 11:25:10 +02:00
Alexander Sibiryakov
5916df64cd
Removing unnecessary import.
2015-04-01 11:24:12 +02:00
Daniel Graña
ec4251afd7
Merge pull request #1089 from drack3800/master
...
[MRG+1] Add test for webclient with POST method and no body given
2015-03-31 17:30:08 -03:00
Alexander Sibiryakov
5864d291d4
Setting documentation.
2015-03-31 11:10:56 +02:00
Alexander Sibiryakov
c1cd019f56
Setting maximum thread pool size in reactor.
2015-03-31 10:56:31 +02:00
Elias Dorneles
0c74821805
Merge pull request #1112 from eliasdorneles/minor-grammar-fixes
...
Some minor grammar fixes
2015-03-27 21:23:55 -03:00
Elias Dorneles
18c5bc75c2
some more minor grammar fixes
2015-03-27 21:00:21 -03:00
Elias Dorneles
7135c4e694
some minor grammar fixes
2015-03-27 17:56:56 -03:00
Nicolás Alejandro Ramírez Quiros
a3702e749f
Merge pull request #1073 from eliasdorneles/add-more-settings-to-template
...
[MRG+1] Adding more settings to project template
2015-03-27 15:37:26 -03:00
Pablo Hoffman
bb4c922d85
Merge pull request #1081 from scrapy/dict-items
...
Allow spiders to return dicts.
2015-03-27 15:19:27 -03:00
Daniel Graña
55a23d102f
Merge pull request #1086 from Curita/response-urljoin
...
Add Response.urljoin() helper
2015-03-27 15:17:54 -03:00
Julia Medina
f4e241a018
Merge pull request #1106 from eliasdorneles/overview-page-improvements
...
[MRG+1] some improvements to overview page
2015-03-27 15:16:33 -03:00
nyov
84edc2ebc7
Add Response.urljoin() testcase
...
and add evaluation of base-url for HtmlResponse.
2015-03-27 05:59:01 +00:00
Mikhail Korobov
39085ae18f
Merge pull request #1098 from nyov/nyov/userconfig
...
[+1 MRG]look in ~/.config/scrapy.cfg for user config
2015-03-27 02:11:35 +05:00
nyov
1134a9cab0
config: look in ~/.config/scrapy.cfg as well
2015-03-26 20:36:14 +00:00
Mikhail Korobov
1ce0425542
Merge pull request #1108 from pbronez/master
...
Converted sel.xpath() calls to response.xpath() in Extracting the data. Fixes GH-1107.
2015-03-27 00:52:17 +05:00
Peter Bronez
475766c73a
Converted sel.xpath() calls to response.xpath() in Extracting the data
2015-03-26 15:34:30 -04:00
Elias Dorneles
4dcecc98f9
moved example data to a better place
2015-03-26 15:45:17 -03:00
Elias Dorneles
7402e27230
fix community link
2015-03-26 15:35:31 -03:00
Elias Dorneles
729861c864
fixing indentation
2015-03-26 15:31:42 -03:00
Elias Dorneles
13d0ecde77
addressing more review comments, to avoid ambiguity on desired reading flow
2015-03-26 15:26:16 -03:00
Elias Dorneles
76e3bf1250
addressing comments from the review plus further editing
2015-03-26 14:26:20 -03:00
Elias Dorneles
8f4a268f37
added bit about async requests, improved phrasing
2015-03-26 12:14:56 -03:00
Elias Dorneles
32423d4a33
some improvements to overview page
2015-03-25 19:27:52 -03:00
Mikhail Korobov
e3e60da6c2
Merge pull request #1105 from ramiro/patch-1
...
Add missing callback arg in jobs topic example.
2015-03-26 03:19:39 +05:00
Ramiro Morales
933dbc6be6
Oops
2015-03-25 18:33:17 -03:00
Ramiro Morales
ca2575001e
Add missing callback arg in jobs topic example.
2015-03-25 18:32:20 -03:00
drack3800
6afd1a78bd
style fix
2015-03-25 20:43:07 +03:00
Daniel Graña
ad36de4e62
Merge pull request #1101 from nyov/nyov/tls-sni
...
handle TLS SNI
2015-03-24 18:21:10 -03:00
nyov
aaeb837db4
handle TLS SNI if we have twisted>=14.0
...
(closes #981 , #1101 )
2015-03-24 10:52:29 +00:00
Mikhail Korobov
5ac91e4883
DOC remove Dynamic Creation of Item Classes section
...
It was a hack, and dicts-as-items cover most use cases.
Dicts don't allow to attach metadata to fields,
but e.g. adding "_meta" key and removing it in a custom serializer
is no worse than creating classes dynamically.
2015-03-23 18:11:35 +05:00
drack3800
549882590b
Fixed bug with no specified Content-Length header by ScrapyHTTPClientFactory for POST request with no given body
2015-03-22 19:31:29 +03:00
drack3800
deb5bb530c
Fixed bug with no specified Content-Length header by ScrapyHTTPClientFactory for POST request with no given body
2015-03-22 19:25:08 +03:00
drack3800
1b6d5a011a
Added webclient test for checking Content-Length header in response for POST request with no given body
2015-03-21 04:02:51 +03:00
Julia Medina
cda3922507
Add Response.urljoin() helper
2015-03-19 19:07:52 -03:00
Pablo Hoffman
c81eefaf81
fix doc links
2015-03-19 17:42:48 -03:00
Elias Dorneles
d7cb2b9a91
making commented code indentation consistent
2015-03-19 15:41:43 -03:00
Mikhail Korobov
8ac397670f
DOC move .. module: declaration to a proper place
2015-03-19 21:41:36 +05:00
Julia Medina
67a85c77f2
Merge branch 'coder46-master' from pull request #647
2015-03-18 23:55:42 -03:00
Faisal Anees
643984e1b4
Updated architecture.rst
...
Added http://krondo.com/blog/?page_id=1327 as a resource
2015-03-18 23:55:22 -03:00
Julia Medina
ff64584876
Merge branch 'ananana-selectorlist-extract-first' from pull request #624
2015-03-18 21:29:49 -03:00
Mikhail Korobov
f16a33f34e
DOC change structure of spider docs:
...
* start with scrapy.Spider, then mention spider arguments,
then describe generic spiders;
* change wording regarding start_urls/start_requests;
* show an example of start_requests vs start_urls;
* show an example of dicts as items;
* as defining Item is an optional step now, docs for Items are
moved below Spider docs.
2015-03-19 05:25:15 +05:00
Mikhail Korobov
817dbc6cbd
DOC mention dicts in documentation; explain better what are Items for
2015-03-19 05:16:14 +05:00
Julia Medina
959aaad205
Document re_first
2015-03-18 21:11:08 -03:00
