scraperdragon
0ba0d85685
Parameters to Request() in wrong order
...
Implied that callback wasn't the first optional unnamed parameter.
2013-10-15 11:50:43 +01:00
Daniel Graña
28999590fa
update release notes
2013-10-14 16:41:04 -02:00
Daniel Graña
ab9462a251
remove more references to libxml2
2013-10-14 16:37:14 -02:00
Daniel Graña
4645f9e03c
Updates docs to reflect unified selectors api
2013-10-14 16:31:20 -02:00
Daniel Graña
bf37f78572
Drop libxml2 selectors backend
2013-10-11 18:02:35 -02:00
Daniel Graña
6d598f0d94
Update selectors docs
2013-10-10 18:24:00 -02:00
Capi Etheriel
bc17e9d412
Adds HtmlCSSSelector and XmlCSSSelector classes, cssselect as optional dependency.
...
Ported .get() from _Element and .text_content() from HTMLMixin
Add CSS selectors to scrapy shell
Documenting CSS Selectors: Constructing selectors
Documenting CSS Selectors: Using Selectors
Make CSS Selectors a default feature.
Adds XPath powers to CSS Selectors and some syntactic sugar.
Removes methods copied over from lxml.html.HtmlMixin.
Updating docs to use new CSS Selector super powers.
Documenting CSS Selectors: Regular Expressions
Moving section after Nesting section, since it mentions it.
Documenting CSS Selectors: Nesting Selectors
Fix XPath specificity in lxml.selector.CSSSelectorMixin.text
Cleaning up unused stuff from cssel.py
Changing the behavior of lxml.selector.CSSSelectorMixin.text.
Concatenating all of the descendant text nodes is more useful
than returning it in pieces (there's xpath() if you need that).
Documenting CSS Selectors: CSS Selector objects
Documenting CSS Selectors: CSSSelectorList objects
Documenting CSS Selectors: HtmlCSSSelector objects
Documenting CSS Selectors: XmlCSSSelector objects
Fixing some documentations typos and errors
Enforcing the 80-char width lines
Tidying up CSS selectors and CSSSelectorMixin objects
Adding some missing references in documentation.
Fixing lxml.selector.CSSSelectorList.text
2013-10-10 18:23:15 -02:00
Daniel Graña
5b5dd679b0
Add 0.18.4 release notes
...
Conflicts:
docs/news.rst
2013-10-10 01:04:35 -02:00
Pablo Hoffman
e1683ddf9b
fix doc typo
2013-10-09 17:24:12 -02:00
Pablo Hoffman
8b9526a8f6
Merge pull request #400 from irgmedeiros/patch-2
...
Update the second code example
2013-10-07 07:57:18 -07:00
Pablo Hoffman
86c6e9433f
remove minor reference to 'scrapy server' command
2013-10-04 14:37:55 -03:00
Daniel Graña
aad90ec5a2
Add 0.18.3 release notes
...
Conflicts:
docs/news.rst
2013-10-03 12:56:25 -03:00
Pablo Hoffman
37c24e01d7
document bindaddress request meta
2013-10-02 17:13:17 -03:00
Pablo Hoffman
a9c3519897
updated required twisted version to 10.0
2013-10-01 14:07:38 -03:00
Rolando Espinoza
d6e3eae527
docs: added section regarding setting up django's settings.
2013-09-30 09:58:10 -04:00
Rolando Espinoza
0cc1d870db
docs: minor tidy up sample code and missing shell prompts.
2013-09-30 09:58:10 -04:00
Loren Davie
8af0e89e85
Corrected typo.
2013-09-29 17:06:46 -04:00
Loren Davie
f49f5724d5
Added dynamic creation of item classes to practices.rst.
2013-09-28 09:00:48 -04:00
irgmedeiros
9b50409986
Update the second code example
...
Update the second code example to reflect the last change in the first example.
2013-09-27 18:22:33 -03:00
irgmedeiros
d9e0fdc9aa
Update practices.rst
...
With this modification scrapy runs the spider with project settings. The previous example ran only with default settings resulting in ignoring all user settings as pipelines for example.
2013-09-27 17:56:30 -03:00
Daniel Graña
265910aae6
Merge pull request #363 from taikano/sitemap_alternate
...
also fetch alternate URLs from sitemaps, see #360
2013-09-26 09:15:02 -07:00
Pablo Hoffman
12280c2a95
fix sphinx references in doc
2013-09-25 15:13:17 -03:00
Pablo Hoffman
fc388f4636
Make ITEM_PIPELINE setting a dict
...
This is for consistency with how spider and downloader middlewares are
defined. ITEM_PIPELINE_BASE was also added and both remain empty.
Backwards compatibility is kept (with a warning) with list-based
ITEM_PIPELINES.
2013-09-23 17:50:43 -03:00
Pablo Hoffman
b1d1a36a1e
add note about enclosing urls with quotes when running from command-line. closes GH-384
2013-09-18 18:01:28 -03:00
cacovsky
71b320914a
Update request-response.rst
...
Fix small doc typo (too many backticks)
2013-09-18 11:45:25 -03:00
Kumara Tharmalingam
bbb0603091
Fixed directory location for dmoz_spider.py file
...
It should be under 'tutorial/spiders' not 'dmoz/spiders'
2013-09-15 21:55:52 -07:00
Daniel Graña
0400b18efa
docs: list lxml as installation prerequisite
2013-09-09 12:44:26 -03:00
Stefan
6994959181
renamed to sitemap_alternate_links and added default value, see #360
2013-09-08 10:38:28 +02:00
Stefan
8ed2d0cda1
improved changes to allow retrieval of alternate links in sitemaps, see #360
2013-09-07 12:56:30 +02:00
Daniel Graña
e6b3ca0180
Add 0.18.2 release notes
2013-09-03 14:29:40 -03:00
Daniel Graña
0f00b1602a
merge 0.18 release notes
2013-08-27 18:47:46 -03:00
Pablo Hoffman
19ff9ac4f9
url/body attributes of Request/Response objects are now immutable
2013-08-23 12:43:22 -03:00
Pablo Hoffman
86230c0ab8
added quantal & raring to support ubuntu releases
2013-08-22 21:49:55 -03:00
Mikhail Korobov
034ffae60f
Recommend Pillow instead of PIL. Closes GH-317.
2013-08-18 00:44:01 +06:00
Berend Iwema
32b6364bcd
#327 - Support STARTTLS / SSL option in email sender
2013-08-14 12:59:01 +02:00
Pablo Hoffman
c0b26e3d49
minor updates to 0.18 release notes
2013-08-14 01:39:44 -03:00
Daniel Graña
ed5b9068d2
fix contributters list format
2013-08-12 11:26:16 -03:00
Daniel Graña
a6693c9a5c
updated release notes and bumped version to 0.18.0
2013-08-09 19:02:28 -03:00
Hart
c00c4d7148
correction to description of example XPath retrieval in overview doc
2013-08-03 17:08:58 -07:00
Hart
0ad01c34d4
fixed typo to parallel fix on 0.16 branch
2013-08-03 17:06:10 -07:00
Rocio Aramberri
d227d530f6
Added COMPRESSION_ENABLED setting to enable or disable the HttpCompressionMiddleware
...
Added COMPRESSION_ENABLE setting to docs
Added COMPRESSION_ENABLED setting to default settings
2013-08-01 11:31:28 -03:00
Dan
1ca31244b0
Fixed ordering of super argument call.
2013-07-16 14:50:10 -04:00
Dan
e12b689c4f
Updated documentation of spider arguments to include required super call.
2013-07-16 14:26:53 -04:00
Mikhail Korobov
1a1c93fafe
tiny FormRequest doc fix
2013-07-15 15:47:34 +06:00
Mikhail Korobov
ac2fadf3ab
DownloaderMiddleware.process_response docs fix
...
"returns an exception" -> "raises an exception"
2013-07-08 19:41:58 +06:00
Mikhail Korobov
39e5da5f66
improve docs for DownloaderMiddleware.process_response
2013-07-08 19:17:29 +06:00
Pablo Hoffman
0f4b70f582
remove no deprecated request_scheduled signal
...
It will be replaced by more accurate scheduler signals (proposal will
come soon)
2013-06-27 11:23:24 -03:00
nramirezuy
bef8ade956
removed request_received and added request_scheduled
2013-06-26 16:45:46 -03:00
Pablo Hoffman
819b2776dd
Merge pull request #326 from berendiwema/master
...
Include example of how to stop the reactor from script
2013-06-25 13:30:07 -07:00
nramirezuy
83b2774354
remove wrong default httpcache
2013-06-25 17:01:29 -03:00
