BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-26 16:23:57 +00:00
Code Releases Wiki Activity
4,019 Commits 39 Branches 119 Tags
Commit Graph

4019 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Rolando Espinoza La fuente
dc666f6e5d PEP8 improvements 
* Space lines around classes/functions.
* Space in inline comments.
* Remove redundant backslash between brackets.
 2013-10-19 15:40:42 -04:00
Mikhail Korobov
62fd5b3213 Merge pull request #430 from alexanderlukanin13/cleanup 
pylint cleanup: unused imports and old-style exceptions
 2013-10-18 16:20:04 -07:00
alexanderlukanin13
105e46205b restored import *, added comment 2013-10-19 00:48:22 +06:00
Daniel Graña
1f17230f17 add testing environment for development versions of w3lib and queuelib 2013-10-18 16:08:27 -02:00
alexanderlukanin13
ddcd571000 pylint cleanup: unused imports and old-style exceptions 2013-10-18 19:23:45 +06:00
Mikhail Korobov
2e8cc281b0 Merge pull request #429 from alexanderlukanin13/bugfix 
wrong variable name
 2013-10-18 02:28:06 -07:00
alexanderlukanin13
192c323a0c wrong variable name 2013-10-18 11:30:48 +06:00
Daniel Graña
757e262267 update requirement for cssselect >= 0.9 2013-10-17 09:33:15 -02:00
Daniel Graña
875b07aef8 fix references to old selector naming in docs 2013-10-17 09:33:15 -02:00
Travis Briggs
3043a5ba37 DownloaderMiddleware docs: Update process_request, proper explanation of IgnoreRequest. 
Also:
* Change terminology to eliminate uses of terms such as "request middleware" to refer to the process_request methods of installed middleware.
* Remove description of "immediate redirection", as it is misleading.

Further changes.
 2013-10-17 00:23:21 +00:00
Mikhail Korobov
086b8a20d4 typo fix in TextResponse docs 2013-10-17 04:50:30 +06:00
Pablo Hoffman
951a9f3f4c Merge pull request #226 from scraperdragon/patch-1 
Parameters to Request() in wrong order
 2013-10-16 13:15:52 -07:00
Daniel Graña
289688e39e Merge pull request #426 from scrapy/selectors-unified 
[MRG] Selectors unified API
 2013-10-16 12:58:12 -07:00
Daniel Graña
1461363809 Replace contenttype references by type 
The type to choose from is the selector type, not the input type. A
content-type doesn't make sense in this context.
 2013-10-16 17:37:25 -02:00
Daniel Graña
155ea08ea1 use sel name for Selector's instances in docs, internals and shell 2013-10-15 15:58:42 -02:00
Daniel Graña
1abb1af0c6 fix typos and wording on selector's introduction 2013-10-15 10:13:43 -02:00
Dragon Dave
a3b711bdea Move callback blob; mention errback 2013-10-15 12:19:42 +01:00
scraperdragon
0ba0d85685 Parameters to Request() in wrong order 
Implied that callback wasn't the first optional unnamed parameter.
 2013-10-15 11:50:43 +01:00
Daniel Graña
28999590fa update release notes 2013-10-14 16:41:04 -02:00
Daniel Graña
ab9462a251 remove more references to libxml2 2013-10-14 16:37:14 -02:00
Daniel Graña
4645f9e03c Updates docs to reflect unified selectors api 2013-10-14 16:31:20 -02:00
Daniel Graña
add3506928 remove internal references to old selector classes and api 2013-10-14 10:51:16 -02:00
Daniel Graña
4e94b38396 port tests to new Selector class 2013-10-14 10:35:02 -02:00
Daniel Graña
e4d6e2eb31 default xpath selector was html 2013-10-11 19:53:32 -02:00
Daniel Graña
a9eb0b74c8 port scrapy shell to unified selector api 2013-10-11 19:53:01 -02:00
Daniel Graña
681af6b258 Remove CSS*Selector classes and port its tests 2013-10-11 18:22:03 -02:00
Daniel Graña
c3d28cc412 working implementaion of unified api 2013-10-11 18:06:27 -02:00
Daniel Graña
bf37f78572 Drop libxml2 selectors backend 2013-10-11 18:02:35 -02:00
Daniel Graña
6d598f0d94 Update selectors docs 2013-10-10 18:24:00 -02:00
Paul Tremberth
ea579bca04 Support only ::text and ::attr(name) + add more CSS selector tests 2013-10-10 18:23:15 -02:00
Paul Tremberth
52e2eb6b70 Adapt to latest cssselect API supporting pseudo-elements 2013-10-10 18:23:15 -02:00
Daniel Graña
4e6967b854 extend css selectors with ":text" and :attribute(<name>) #176 2013-10-10 18:23:15 -02:00
Daniel Graña
b38ac27eee rename XPathSelectorList as SelectorList #176 2013-10-10 18:23:15 -02:00
Capi Etheriel
bc17e9d412 Adds HtmlCSSSelector and XmlCSSSelector classes, cssselect as optional dependency. 
Ported .get() from _Element and .text_content() from HTMLMixin

Add CSS selectors to scrapy shell

Documenting CSS Selectors: Constructing selectors

Documenting CSS Selectors: Using Selectors

Make CSS Selectors a default feature.

Adds XPath powers to CSS Selectors and some syntactic sugar.

Removes methods copied over from lxml.html.HtmlMixin.

Updating docs to use new CSS Selector super powers.

Documenting CSS Selectors: Regular Expressions

Moving section after Nesting section, since it mentions it.

Documenting CSS Selectors: Nesting Selectors

Fix XPath specificity in lxml.selector.CSSSelectorMixin.text

Cleaning up unused stuff from cssel.py

Changing the behavior of lxml.selector.CSSSelectorMixin.text.

Concatenating all of the descendant text nodes is more useful
than returning it in pieces (there's xpath() if you need that).

Documenting CSS Selectors: CSS Selector objects

Documenting CSS Selectors: CSSSelectorList objects

Documenting CSS Selectors: HtmlCSSSelector objects

Documenting CSS Selectors: XmlCSSSelector objects

Fixing some documentations typos and errors

Enforcing the 80-char width lines

Tidying up CSS selectors and CSSSelectorMixin objects

Adding some missing references in documentation.

Fixing lxml.selector.CSSSelectorList.text
 2013-10-10 18:23:15 -02:00
Daniel Graña
8bf3284ebf Merge pull request #380 from dangra/dont-log-ignorerequest 
Dont log IgnoreRequest exception as download failure
 2013-10-10 11:53:08 -07:00
Pablo Hoffman
1a2db089ff Merge pull request #418 from nramirezuy/engine-multispider 
engine multispider support removed
 2013-10-10 10:31:13 -07:00
Daniel Graña
5b5dd679b0 Add 0.18.4 release notes 
Conflicts:
	docs/news.rst
 2013-10-10 01:04:35 -02:00
Daniel Graña
aa6fb7daaa IPython refuses to update the namespace. fix #396 
IPython embedding code borrowed from https://github.com/mitsuhiko/werkzeug/pull/85
 2013-10-10 00:57:14 -02:00
Daniel Graña
7b1288ba54 Fix AlreadyCalledError replacing a request in shell command. closes #407 2013-10-09 23:50:35 -02:00
Daniel Graña
5eb429999e Fix start_requests lazyness and early hangs 
- Removes new public methods added by #330 to Crawler and CrawlerProcess
- Add test for start_requests lazy evaluation
- Fix and test hangs when start_requests erroed before returning the generator
- Add test when start_requests fails while generating requests
- Simplify Crawler and CrawlerProcess implementation taking in count
  that only one spider can be attached per Crawler. As required by SEP-019
- "scrapy settings" command do not require starting a Crawler anymore
 2013-10-09 22:58:10 -02:00
Pablo Hoffman
2bd5002fb4 Merge pull request #416 from redapple/attrnamespaces 
.remove_namespaces(): remove namespaces on elements' attributes
 2013-10-09 15:34:08 -07:00
Pablo Hoffman
e1683ddf9b fix doc typo 2013-10-09 17:24:12 -02:00
nramirezuy
a84e5f806c engine multispider support removed 2013-10-09 15:02:12 -02:00
Pablo Hoffman
ce82d22c9e Merge pull request #423 from nramirezuy/defaultheaders-multispider 
defaultheaders multi spider support removed
 2013-10-09 09:03:33 -07:00
nramirezuy
332bf3b6b7 defaultheaders multi spider support removed 2013-10-09 13:50:15 -02:00
Pablo Hoffman
7ec01799d5 Merge pull request #419 from nramirezuy/robotstxt-multispider 
robotstxt mid multi spider support removed
 2013-10-09 07:43:11 -07:00
Pablo Hoffman
9ad736d7ce Merge pull request #420 from nramirezuy/httpauth-multispider 
httpauth mid multispider support removed
 2013-10-08 21:06:49 -07:00
Pablo Hoffman
27d3cbb64f Merge pull request #421 from nramirezuy/useragent-multispider 
useragent mid multi spider support removed
 2013-10-08 21:06:13 -07:00
Pablo Hoffman
37b01a229c Merge pull request #422 from nramirezuy/downloadtimeout-multispider 
downloadtimeout mid multi spider support removed
 2013-10-08 21:05:06 -07:00
nramirezuy
1d09b6e1f2 downloadtimeout mid multi spider support removed 2013-10-08 19:03:26 -02:00