BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-26 17:24:38 +00:00
Code Releases Wiki Activity
3,889 Commits 39 Branches 119 Tags
Commit Graph

3889 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mikhail Korobov
f80f10ae7e Merge pull request #452 from alexanderlukanin13/python3 
PY3: scrapy.__version__, NoneType, urlparse_monkeypatches
 2013-11-04 00:54:02 -08:00
alexanderlukanin13
6c7292a08e python3: scrapy.__version__, NoneType, urlparse_monkeypatches 2013-11-03 22:20:33 +06:00
Rolando Espinoza La fuente
d1b912890b Merge branch 'master' into import-module 2013-10-30 21:07:53 -04:00
Pablo Hoffman
fa245af6d2 Merge pull request #448 from dangra/drop-py26 
Drop Python 2.6 support
 2013-10-30 05:53:06 -07:00
Daniel Graña
2df8156431 Drop Python 2.6 support 2013-10-29 13:44:00 -02:00
Rolando Espinoza La fuente
10e22aa5fb Replaced remaning __import__(module) calls. 
This commit replaces the statements __import__(module) as the previous
replaced the statements __import__(module, {}, {}, ['']).

At first I thought leaving the single-argument calls, but perhaps it's
better to be strict rather than having exceptions to the rule in this
case.
 2013-10-27 19:10:25 -04:00
Rolando Espinoza La fuente
343f997ed6 replaced __import__ by importlib.import_module. 
Since python 2.7, importlib.import_module is the recommended way to
import modules programmatically.

From __import__'s doc:

    Import a module. Because this function is meant for use by the
Python
    interpreter and not for general use it is better to use
    importlib.import_module() to programmatically import a module.
 2013-10-27 18:33:51 -04:00
Daniel Graña
bd79b6e1d3 debian package requires python-cssselect 2013-10-24 16:47:46 -02:00
Mikhail Korobov
247b2ad596 Merge pull request #431 from alexanderlukanin13/syntax 
Python 3 compatible syntax: print, except, raise, octal numbers; removed...
 2013-10-22 10:38:45 -07:00
alexanderlukanin13
6b598476eb print_function in xlib 2013-10-22 22:49:18 +06:00
Mikhail Korobov
3d4904be16 Merge pull request #436 from alexanderlukanin13/syntax-next 
Using next() for Python 3 compatibility; ordereddict.py converted from DOS to Unix
 2013-10-22 08:35:09 -07:00
Mikhail Korobov
bf6ef29182 Merge pull request #435 from alexanderlukanin13/syntax-map 
Converted map() to list comprehension/generator for Python3 compatibility
 2013-10-22 07:37:57 -07:00
alexanderlukanin13
a958cb2c5d Using next() for Python 3 compatibility 2013-10-22 13:54:22 +06:00
alexanderlukanin13
6b35166de6 Converted map() to list comprehension or generator for Python 3 compatibility 2013-10-22 13:18:26 +06:00
Pablo Hoffman
911c8082b0 simplified description of crawl command 2013-10-21 14:42:51 -02:00
Pablo Hoffman
e8ee449a2a Merge pull request #432 from darkrho/crawl-url 
Removed URL reference in crawl command and .tld suffix in docs for spider names
 2013-10-21 09:40:58 -07:00
alexanderlukanin13
ae4a61a2d5 Added "from __future__ import print_function" in all relevant places except xlib 2013-10-20 11:55:07 +06:00
Rolando Espinoza La fuente
34543c2b2e DOCS removed .tld suffix for spider names for the sake of consistency. 2013-10-19 23:03:20 -04:00
Rolando Espinoza La fuente
c2ced9a234 Removed reference to URL in crawl command as it's no longer supported. 2013-10-19 22:57:37 -04:00
alexanderlukanin13
d381a35732 Python 3 compatible syntax: print, except, raise, octal numbers; removed Python 2.2 boolean compatibility code in xlib/pydispatch/dispatcher.py 2013-10-20 01:59:35 +06:00
Rolando Espinoza La fuente
dc666f6e5d PEP8 improvements 
* Space lines around classes/functions.
* Space in inline comments.
* Remove redundant backslash between brackets.
 2013-10-19 15:40:42 -04:00
Mikhail Korobov
62fd5b3213 Merge pull request #430 from alexanderlukanin13/cleanup 
pylint cleanup: unused imports and old-style exceptions
 2013-10-18 16:20:04 -07:00
alexanderlukanin13
105e46205b restored import *, added comment 2013-10-19 00:48:22 +06:00
Daniel Graña
1f17230f17 add testing environment for development versions of w3lib and queuelib 2013-10-18 16:08:27 -02:00
alexanderlukanin13
ddcd571000 pylint cleanup: unused imports and old-style exceptions 2013-10-18 19:23:45 +06:00
Mikhail Korobov
2e8cc281b0 Merge pull request #429 from alexanderlukanin13/bugfix 
wrong variable name
 2013-10-18 02:28:06 -07:00
alexanderlukanin13
192c323a0c wrong variable name 2013-10-18 11:30:48 +06:00
Daniel Graña
757e262267 update requirement for cssselect >= 0.9 2013-10-17 09:33:15 -02:00
Daniel Graña
875b07aef8 fix references to old selector naming in docs 2013-10-17 09:33:15 -02:00
Travis Briggs
3043a5ba37 DownloaderMiddleware docs: Update process_request, proper explanation of IgnoreRequest. 
Also:
* Change terminology to eliminate uses of terms such as "request middleware" to refer to the process_request methods of installed middleware.
* Remove description of "immediate redirection", as it is misleading.

Further changes.
 2013-10-17 00:23:21 +00:00
Mikhail Korobov
086b8a20d4 typo fix in TextResponse docs 2013-10-17 04:50:30 +06:00
Pablo Hoffman
951a9f3f4c Merge pull request #226 from scraperdragon/patch-1 
Parameters to Request() in wrong order
 2013-10-16 13:15:52 -07:00
Daniel Graña
289688e39e Merge pull request #426 from scrapy/selectors-unified 
[MRG] Selectors unified API
 2013-10-16 12:58:12 -07:00
Daniel Graña
1461363809 Replace contenttype references by type 
The type to choose from is the selector type, not the input type. A
content-type doesn't make sense in this context.
 2013-10-16 17:37:25 -02:00
Daniel Graña
155ea08ea1 use sel name for Selector's instances in docs, internals and shell 2013-10-15 15:58:42 -02:00
Daniel Graña
1abb1af0c6 fix typos and wording on selector's introduction 2013-10-15 10:13:43 -02:00
Dragon Dave
a3b711bdea Move callback blob; mention errback 2013-10-15 12:19:42 +01:00
scraperdragon
0ba0d85685 Parameters to Request() in wrong order 
Implied that callback wasn't the first optional unnamed parameter.
 2013-10-15 11:50:43 +01:00
Daniel Graña
28999590fa update release notes 2013-10-14 16:41:04 -02:00
Daniel Graña
ab9462a251 remove more references to libxml2 2013-10-14 16:37:14 -02:00
Daniel Graña
4645f9e03c Updates docs to reflect unified selectors api 2013-10-14 16:31:20 -02:00
Daniel Graña
add3506928 remove internal references to old selector classes and api 2013-10-14 10:51:16 -02:00
Daniel Graña
4e94b38396 port tests to new Selector class 2013-10-14 10:35:02 -02:00
Daniel Graña
e4d6e2eb31 default xpath selector was html 2013-10-11 19:53:32 -02:00
Daniel Graña
a9eb0b74c8 port scrapy shell to unified selector api 2013-10-11 19:53:01 -02:00
Daniel Graña
681af6b258 Remove CSS*Selector classes and port its tests 2013-10-11 18:22:03 -02:00
Daniel Graña
c3d28cc412 working implementaion of unified api 2013-10-11 18:06:27 -02:00
Daniel Graña
bf37f78572 Drop libxml2 selectors backend 2013-10-11 18:02:35 -02:00
Daniel Graña
6d598f0d94 Update selectors docs 2013-10-10 18:24:00 -02:00
Paul Tremberth
ea579bca04 Support only ::text and ::attr(name) + add more CSS selector tests 2013-10-10 18:23:15 -02:00