1
0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-25 04:04:21 +00:00

58 Commits

Author SHA1 Message Date
Panayiotis Lipiridis
3f8be374c3 Github >> GitHub 2016-05-09 16:15:18 +02:00
Paul Tremberth
1075587dbd Add support for Sphinx 1.4
See http://www.sphinx-doc.org/en/stable/changes.html#release-1-4-released-mar-28-2016

sphinx_rtd_theme has become optional, needs to be added to reqs

https://github.com/sphinx-doc/sphinx/pull/2320 changes node entries tuples
to 5 values instead of 4

`sh` syntax highlighting added very locally in selectors.rst
because of this warning/error with Sphinx 1.4:

```
Warning, treated as error:
/home/paul/src/scrapy/docs/topics/selectors.rst:743:
WARNING: Could not lex literal_block as "python". Highlighting skipped.
```
2016-03-30 14:40:52 +02:00
Lele
7082454f2a Changed sel. to response. for clarity
Changed sel. to response. to comply with the rest of the examples in the same section, to avoid confusion.
2016-03-28 05:27:15 +05:00
nyov
5876b9aa30 Update documentation links 2016-03-03 16:28:33 +00:00
David Chen
0025d5a943 Fixed minor grammar issues. 2015-11-16 07:30:17 +08:00
hoatle
2869cf8dde fix another invalid xpath error 2015-10-07 16:03:43 +07:00
Hoat Le
4e66955411 fix ValueError: Invalid XPath: //div/[id="not-exists"]/text() on selectors.rst
>>> response.xpath('//div/[id="not-exists"]/text()').extract_first() is None
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/vagrant/.virtualenvs/scrapy/lib/python2.7/site-packages/scrapy/http/response/text.py", line 109, in xpath
    return self.selector.xpath(query)
  File "/home/vagrant/.virtualenvs/scrapy/lib/python2.7/site-packages/scrapy/selector/unified.py", line 100, in xpath
    raise ValueError(msg if six.PY3 else msg.encode("unicode_escape"))
ValueError: Invalid XPath: //div/[id="not-exists"]/text()
2015-10-07 15:43:02 +07:00
Bryan Crowe
6a4c475e87 Fix a couple typos 2015-06-11 19:47:30 -04:00
Mikhail Korobov
cc2258b2bb Merge pull request #1145 from bosnj/master
[MRG+1] default return value for extract_first
2015-05-21 22:03:54 +05:00
Julia Medina
c271d8f0b1 Title underline too short in docs/topics/selectors.rst 2015-05-09 05:20:54 -03:00
bosnj
8ae05478be added docs and test case, fixed handling empty string vs None 2015-05-04 21:22:17 +02:00
Elias Dorneles
3d3633f3d2 favoring web scraping over screen scraping in the descriptions 2015-04-25 11:20:20 -03:00
bagratte
beea9267a1 minor corrections in documentation. 2015-04-18 19:48:25 +04:00
Daniel Graña
f71175a449 More replaces of references to old sel shortcut 2015-04-12 21:56:23 -03:00
Julia Medina
959aaad205 Document re_first 2015-03-18 21:11:08 -03:00
Mateusz Golewski
127c6c694a Fix extract_first() docs 2015-03-18 21:11:08 -03:00
Mateusz Golewski
012211accd Add docs for extract_first() 2015-03-18 21:11:08 -03:00
Shadab Zafar
5a58d64131 Fix some redirection links in documentation
Fixes #606
2015-03-18 19:41:26 -03:00
Elias Dorneles
c298eaf164 removes weird indentation in the shell results 2014-07-23 20:42:36 -03:00
Elias Dorneles
514e8582d0 improved explanations, clarified blog post as source, added link for XPath string functions in the spec 2014-07-23 19:43:57 -03:00
Elias Dorneles
80de43ddab adding some xpath tips to selectors docs 2014-07-22 23:20:53 -03:00
Mikhail Korobov
bdea071af3 DOC selectors.rst cleanup 2014-04-24 22:54:17 +06:00
ncp1113
5f18816428 for loops have to have a : at the end of the line
changed 3 instances
2014-04-24 22:50:21 +06:00
Daniel Graña
18412d75b5 Ammend example nesting selectors 2014-04-24 10:32:17 -03:00
Daniel Graña
1117687c47 update docs 2014-04-23 23:39:58 -03:00
Mikhail Korobov
2d3803672b DOC use top-level shortcuts in docs 2014-04-15 01:09:35 +06:00
Julia Medina
80081054a2 Fix broken links in documentation 2014-04-09 18:57:52 -03:00
Paul Tremberth
827c0cf51f Rename "regexp" prefix to "re" 2014-01-15 15:00:25 +01:00
Paul Tremberth
88c8a523a7 Add warning in docs on performance when using EXSLT regexp functions 2014-01-15 12:52:10 +01:00
Paul Tremberth
a3eba68aca Drop EXSLT strings and math extensions 2014-01-15 12:28:25 +01:00
Paul Tremberth
2cc26e6f56 Fix typo error 2014-01-14 13:09:18 +01:00
Paul Tremberth
29fc9f3466 Update selectors documentation and tests 2014-01-14 12:56:37 +01:00
Mikhail Korobov
a27d91f0a6 Rename BaseSpider to Spider. See GH-495. 2013-12-30 19:46:41 +06:00
Daniel Graña
875b07aef8 fix references to old selector naming in docs 2013-10-17 09:33:15 -02:00
Daniel Graña
1461363809 Replace contenttype references by type
The type to choose from is the selector type, not the input type. A
content-type doesn't make sense in this context.
2013-10-16 17:37:25 -02:00
Daniel Graña
155ea08ea1 use sel name for Selector's instances in docs, internals and shell 2013-10-15 15:58:42 -02:00
Daniel Graña
4645f9e03c Updates docs to reflect unified selectors api 2013-10-14 16:31:20 -02:00
Daniel Graña
bf37f78572 Drop libxml2 selectors backend 2013-10-11 18:02:35 -02:00
Daniel Graña
6d598f0d94 Update selectors docs 2013-10-10 18:24:00 -02:00
Capi Etheriel
bc17e9d412 Adds HtmlCSSSelector and XmlCSSSelector classes, cssselect as optional dependency.
Ported .get() from _Element and .text_content() from HTMLMixin

Add CSS selectors to scrapy shell

Documenting CSS Selectors: Constructing selectors

Documenting CSS Selectors: Using Selectors

Make CSS Selectors a default feature.

Adds XPath powers to CSS Selectors and some syntactic sugar.

Removes methods copied over from lxml.html.HtmlMixin.

Updating docs to use new CSS Selector super powers.

Documenting CSS Selectors: Regular Expressions

Moving section after Nesting section, since it mentions it.

Documenting CSS Selectors: Nesting Selectors

Fix XPath specificity in lxml.selector.CSSSelectorMixin.text

Cleaning up unused stuff from cssel.py

Changing the behavior of lxml.selector.CSSSelectorMixin.text.

Concatenating all of the descendant text nodes is more useful
than returning it in pieces (there's xpath() if you need that).

Documenting CSS Selectors: CSS Selector objects

Documenting CSS Selectors: CSSSelectorList objects

Documenting CSS Selectors: HtmlCSSSelector objects

Documenting CSS Selectors: XmlCSSSelector objects

Fixing some documentations typos and errors

Enforcing the 80-char width lines

Tidying up CSS selectors and CSSSelectorMixin objects

Adding some missing references in documentation.

Fixing lxml.selector.CSSSelectorList.text
2013-10-10 18:23:15 -02:00
Shane Evans
5c2a82f1f7 fix typo 2013-03-17 19:34:55 +00:00
Pablo Hoffman
6ab8afb992 improve documentation about removing namespaces 2013-01-18 12:35:30 -02:00
Pablo Hoffman
1ba04b1fc3 added remove_namespaces() method to XmlXPathSelector objects 2013-01-18 12:20:03 -02:00
stav
303e13f616 selector documentation typos 2012-09-18 12:56:52 -05:00
Pablo Hoffman
a874964ad4 renamed 'XPath Selectors' title to just 'Selectors' 2012-09-13 15:24:44 -03:00
Pablo Hoffman
179e3810dc fixed links to doc. closes #150 2012-06-24 01:00:33 -03:00
Pablo Hoffman
a59bfb539d * Added lxml backend for XPath selectors. Closes #147
* Added new setting (SELECTORS_BACKEND) to choose which backend to use
* Deprecated the extract_unquoted() function from selectors
* Made libxml2 optional by adding a dummy selector backend. Closes #260

--HG--
rename : scrapy/tests/test_selector.py => scrapy/tests/test_selector_libxml2.py
2010-10-25 14:47:10 -02:00
Pablo Hoffman
9aefa242d5 Applied documentation patch provided by Lucian Ursu (closes #207) 2010-08-21 01:26:35 -03:00
Pablo Hoffman
34554da201 Deprecated scrapy-ctl.py command in favour of simpler "scrapy" command. Closes #199. Also updated documenation accordingly and added convenient scrapy.bat script for running from Windows.
--HG--
rename : debian/scrapy-ctl.1 => debian/scrapy.1
rename : docs/topics/scrapy-ctl.rst => docs/topics/cmdline.rst
2010-08-18 19:48:32 -03:00
Pablo Hoffman
33b53c59d5 moved scrapy.xpath to scrapy.selector
--HG--
rename : scrapy/xpath/__init__.py => scrapy/selector/__init__.py
rename : scrapy/xpath/document.py => scrapy/selector/document.py
rename : scrapy/xpath/factories.py => scrapy/selector/factories.py
2009-08-19 21:50:52 -03:00