BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-28 08:43:41 +00:00
Code Releases Wiki Activity
3,981 Commits 39 Branches 119 Tags
Commit Graph

3981 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mikhail Korobov
247b2ad596 Merge pull request #431 from alexanderlukanin13/syntax 
Python 3 compatible syntax: print, except, raise, octal numbers; removed...
 2013-10-22 10:38:45 -07:00
alexanderlukanin13
6b598476eb print_function in xlib 2013-10-22 22:49:18 +06:00
Mikhail Korobov
3d4904be16 Merge pull request #436 from alexanderlukanin13/syntax-next 
Using next() for Python 3 compatibility; ordereddict.py converted from DOS to Unix
 2013-10-22 08:35:09 -07:00
Mikhail Korobov
bf6ef29182 Merge pull request #435 from alexanderlukanin13/syntax-map 
Converted map() to list comprehension/generator for Python3 compatibility
 2013-10-22 07:37:57 -07:00
alexanderlukanin13
a958cb2c5d Using next() for Python 3 compatibility 2013-10-22 13:54:22 +06:00
alexanderlukanin13
6b35166de6 Converted map() to list comprehension or generator for Python 3 compatibility 2013-10-22 13:18:26 +06:00
Pablo Hoffman
911c8082b0 simplified description of crawl command 2013-10-21 14:42:51 -02:00
Pablo Hoffman
e8ee449a2a Merge pull request #432 from darkrho/crawl-url 
Removed URL reference in crawl command and .tld suffix in docs for spider names
 2013-10-21 09:40:58 -07:00
alexanderlukanin13
ae4a61a2d5 Added "from __future__ import print_function" in all relevant places except xlib 2013-10-20 11:55:07 +06:00
Rolando Espinoza La fuente
34543c2b2e DOCS removed .tld suffix for spider names for the sake of consistency. 2013-10-19 23:03:20 -04:00
Rolando Espinoza La fuente
c2ced9a234 Removed reference to URL in crawl command as it's no longer supported. 2013-10-19 22:57:37 -04:00
alexanderlukanin13
d381a35732 Python 3 compatible syntax: print, except, raise, octal numbers; removed Python 2.2 boolean compatibility code in xlib/pydispatch/dispatcher.py 2013-10-20 01:59:35 +06:00
Rolando Espinoza La fuente
dc666f6e5d PEP8 improvements 
* Space lines around classes/functions.
* Space in inline comments.
* Remove redundant backslash between brackets.
 2013-10-19 15:40:42 -04:00
Mikhail Korobov
62fd5b3213 Merge pull request #430 from alexanderlukanin13/cleanup 
pylint cleanup: unused imports and old-style exceptions
 2013-10-18 16:20:04 -07:00
alexanderlukanin13
105e46205b restored import *, added comment 2013-10-19 00:48:22 +06:00
Daniel Graña
1f17230f17 add testing environment for development versions of w3lib and queuelib 2013-10-18 16:08:27 -02:00
alexanderlukanin13
ddcd571000 pylint cleanup: unused imports and old-style exceptions 2013-10-18 19:23:45 +06:00
Mikhail Korobov
2e8cc281b0 Merge pull request #429 from alexanderlukanin13/bugfix 
wrong variable name
 2013-10-18 02:28:06 -07:00
alexanderlukanin13
192c323a0c wrong variable name 2013-10-18 11:30:48 +06:00
Daniel Graña
757e262267 update requirement for cssselect >= 0.9 2013-10-17 09:33:15 -02:00
Daniel Graña
875b07aef8 fix references to old selector naming in docs 2013-10-17 09:33:15 -02:00
Travis Briggs
3043a5ba37 DownloaderMiddleware docs: Update process_request, proper explanation of IgnoreRequest. 
Also:
* Change terminology to eliminate uses of terms such as "request middleware" to refer to the process_request methods of installed middleware.
* Remove description of "immediate redirection", as it is misleading.

Further changes.
 2013-10-17 00:23:21 +00:00
Mikhail Korobov
086b8a20d4 typo fix in TextResponse docs 2013-10-17 04:50:30 +06:00
Pablo Hoffman
951a9f3f4c Merge pull request #226 from scraperdragon/patch-1 
Parameters to Request() in wrong order
 2013-10-16 13:15:52 -07:00
Daniel Graña
289688e39e Merge pull request #426 from scrapy/selectors-unified 
[MRG] Selectors unified API
 2013-10-16 12:58:12 -07:00
Daniel Graña
1461363809 Replace contenttype references by type 
The type to choose from is the selector type, not the input type. A
content-type doesn't make sense in this context.
 2013-10-16 17:37:25 -02:00
Daniel Graña
155ea08ea1 use sel name for Selector's instances in docs, internals and shell 2013-10-15 15:58:42 -02:00
Daniel Graña
1abb1af0c6 fix typos and wording on selector's introduction 2013-10-15 10:13:43 -02:00
Dragon Dave
a3b711bdea Move callback blob; mention errback 2013-10-15 12:19:42 +01:00
scraperdragon
0ba0d85685 Parameters to Request() in wrong order 
Implied that callback wasn't the first optional unnamed parameter.
 2013-10-15 11:50:43 +01:00
Daniel Graña
28999590fa update release notes 2013-10-14 16:41:04 -02:00
Daniel Graña
ab9462a251 remove more references to libxml2 2013-10-14 16:37:14 -02:00
Daniel Graña
4645f9e03c Updates docs to reflect unified selectors api 2013-10-14 16:31:20 -02:00
Daniel Graña
add3506928 remove internal references to old selector classes and api 2013-10-14 10:51:16 -02:00
Daniel Graña
4e94b38396 port tests to new Selector class 2013-10-14 10:35:02 -02:00
Daniel Graña
e4d6e2eb31 default xpath selector was html 2013-10-11 19:53:32 -02:00
Daniel Graña
a9eb0b74c8 port scrapy shell to unified selector api 2013-10-11 19:53:01 -02:00
Daniel Graña
681af6b258 Remove CSS*Selector classes and port its tests 2013-10-11 18:22:03 -02:00
Daniel Graña
c3d28cc412 working implementaion of unified api 2013-10-11 18:06:27 -02:00
Daniel Graña
bf37f78572 Drop libxml2 selectors backend 2013-10-11 18:02:35 -02:00
Daniel Graña
6d598f0d94 Update selectors docs 2013-10-10 18:24:00 -02:00
Paul Tremberth
ea579bca04 Support only ::text and ::attr(name) + add more CSS selector tests 2013-10-10 18:23:15 -02:00
Paul Tremberth
52e2eb6b70 Adapt to latest cssselect API supporting pseudo-elements 2013-10-10 18:23:15 -02:00
Daniel Graña
4e6967b854 extend css selectors with ":text" and :attribute(<name>) #176 2013-10-10 18:23:15 -02:00
Daniel Graña
b38ac27eee rename XPathSelectorList as SelectorList #176 2013-10-10 18:23:15 -02:00
Capi Etheriel
bc17e9d412 Adds HtmlCSSSelector and XmlCSSSelector classes, cssselect as optional dependency. 
Ported .get() from _Element and .text_content() from HTMLMixin

Add CSS selectors to scrapy shell

Documenting CSS Selectors: Constructing selectors

Documenting CSS Selectors: Using Selectors

Make CSS Selectors a default feature.

Adds XPath powers to CSS Selectors and some syntactic sugar.

Removes methods copied over from lxml.html.HtmlMixin.

Updating docs to use new CSS Selector super powers.

Documenting CSS Selectors: Regular Expressions

Moving section after Nesting section, since it mentions it.

Documenting CSS Selectors: Nesting Selectors

Fix XPath specificity in lxml.selector.CSSSelectorMixin.text

Cleaning up unused stuff from cssel.py

Changing the behavior of lxml.selector.CSSSelectorMixin.text.

Concatenating all of the descendant text nodes is more useful
than returning it in pieces (there's xpath() if you need that).

Documenting CSS Selectors: CSS Selector objects

Documenting CSS Selectors: CSSSelectorList objects

Documenting CSS Selectors: HtmlCSSSelector objects

Documenting CSS Selectors: XmlCSSSelector objects

Fixing some documentations typos and errors

Enforcing the 80-char width lines

Tidying up CSS selectors and CSSSelectorMixin objects

Adding some missing references in documentation.

Fixing lxml.selector.CSSSelectorList.text
 2013-10-10 18:23:15 -02:00
Daniel Graña
8bf3284ebf Merge pull request #380 from dangra/dont-log-ignorerequest 
Dont log IgnoreRequest exception as download failure
 2013-10-10 11:53:08 -07:00
Pablo Hoffman
1a2db089ff Merge pull request #418 from nramirezuy/engine-multispider 
engine multispider support removed
 2013-10-10 10:31:13 -07:00
Daniel Graña
5b5dd679b0 Add 0.18.4 release notes 
Conflicts:
	docs/news.rst
 2013-10-10 01:04:35 -02:00
Daniel Graña
aa6fb7daaa IPython refuses to update the namespace. fix #396 
IPython embedding code borrowed from https://github.com/mitsuhiko/werkzeug/pull/85
 2013-10-10 00:57:14 -02:00