BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-28 22:37:44 +00:00
Code Releases Wiki Activity
4,157 Commits 39 Branches 119 Tags
Commit Graph

4157 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
olveyra
755b9ba5a4 telnet client: fix unexisting reference to engine.slots 2013-11-19 04:52:24 +01:00
Pablo Hoffman
afe6eaa2fe Merge pull request #460 from tntC4stl3/master 
duplicate 'use' in line 87
 2013-11-15 04:10:49 -08:00
tntC4stl3
b51d5d81e4 duplicate 'use' in line 87 2013-11-15 13:56:44 +08:00
Daniel Graña
c74903f9da process_parallel was leaking the failures on its internal deferreds. closes #458 
DeferredList implemented cancellation in Twisted 13.2.0 by holding a
reference to the affected deferreds objects, if a deferred errored the
result was propagated to the DeferredList but still referenced by the
original deferred and nobody was consuming it.

The tests started to fail because the reference from DeferredList
prevented the underlining deferred from been collected before the test
finish invalidating the effect of self.flushedLoggedErrors() call.
 2013-11-09 02:12:52 -02:00
Daniel Graña
04ff7ecebf improve 0.20 release notes 
Conflicts:
	docs/news.rst
 2013-11-08 17:45:03 -02:00
Daniel Graña
3d18a3c49e bumped version to 0.21.0 0.21.0 2013-11-08 17:09:00 -02:00
Daniel Graña
d0980e5c9b Merge 0.20 release notes 2013-11-08 17:06:10 -02:00
Daniel Graña
60516123a5 Merge branch 'travix-toxed' 
Conflicts:
	requirements.txt
 2013-11-07 11:29:58 -02:00
Daniel Graña
d29791d7ab building Pillow with pypy requries dev headers 2013-11-07 10:58:08 -02:00
Daniel Graña
971f60d796 TOXENV is tox supported env 2013-11-07 10:58:08 -02:00
Daniel Graña
ecfa743105 install updated pypy from ppa 2013-11-07 10:58:08 -02:00
Daniel Graña
fabb351097 map travis-ci matrix to tox environments 2013-11-07 10:58:08 -02:00
Daniel Graña
bc7fa61136 Django 1.6 form validation errors now include ValidationError exception instances instead of just strings 2013-11-07 10:36:39 -02:00
Rolando Espinoza La fuente
6f5423aebd Replaced remaning __import__(module) calls. 
This commit replaces the statements __import__(module) as the previous
replaced the statements __import__(module, {}, {}, ['']).

At first I thought leaving the single-argument calls, but perhaps it's
better to be strict rather than having exceptions to the rule in this
case.
 2013-11-07 10:36:39 -02:00
Rolando Espinoza La fuente
6b1760d7a1 replaced __import__ by importlib.import_module. 
Since python 2.7, importlib.import_module is the recommended way to
import modules programmatically.

From __import__'s doc:

    Import a module. Because this function is meant for use by the
Python
    interpreter and not for general use it is better to use
    importlib.import_module() to programmatically import a module.
 2013-11-07 10:36:39 -02:00
Daniel Graña
b6bed44c2b Django 1.6 form validation errors now include ValidationError exception instances instead of just strings 2013-11-07 02:32:10 -02:00
Daniel Graña
b78e76108f Merge pull request #445 from darkrho/import-module 
Use `importlib.import_module` instead of `__import__`
 2013-11-06 20:14:27 -08:00
Daniel Graña
2318c56f14 shutdown the active crawler on SIGINT. fixes #450 2013-11-05 00:02:17 -02:00
Mikhail Korobov
f80f10ae7e Merge pull request #452 from alexanderlukanin13/python3 
PY3: scrapy.__version__, NoneType, urlparse_monkeypatches
 2013-11-04 00:54:02 -08:00
alexanderlukanin13
6c7292a08e python3: scrapy.__version__, NoneType, urlparse_monkeypatches 2013-11-03 22:20:33 +06:00
Rolando Espinoza La fuente
d1b912890b Merge branch 'master' into import-module 2013-10-30 21:07:53 -04:00
Pablo Hoffman
fa245af6d2 Merge pull request #448 from dangra/drop-py26 
Drop Python 2.6 support
 2013-10-30 05:53:06 -07:00
Daniel Graña
2df8156431 Drop Python 2.6 support 2013-10-29 13:44:00 -02:00
Rolando Espinoza La fuente
10e22aa5fb Replaced remaning __import__(module) calls. 
This commit replaces the statements __import__(module) as the previous
replaced the statements __import__(module, {}, {}, ['']).

At first I thought leaving the single-argument calls, but perhaps it's
better to be strict rather than having exceptions to the rule in this
case.
 2013-10-27 19:10:25 -04:00
Rolando Espinoza La fuente
343f997ed6 replaced __import__ by importlib.import_module. 
Since python 2.7, importlib.import_module is the recommended way to
import modules programmatically.

From __import__'s doc:

    Import a module. Because this function is meant for use by the
Python
    interpreter and not for general use it is better to use
    importlib.import_module() to programmatically import a module.
 2013-10-27 18:33:51 -04:00
Daniel Graña
bd79b6e1d3 debian package requires python-cssselect 2013-10-24 16:47:46 -02:00
Mikhail Korobov
247b2ad596 Merge pull request #431 from alexanderlukanin13/syntax 
Python 3 compatible syntax: print, except, raise, octal numbers; removed...
 2013-10-22 10:38:45 -07:00
alexanderlukanin13
6b598476eb print_function in xlib 2013-10-22 22:49:18 +06:00
Mikhail Korobov
3d4904be16 Merge pull request #436 from alexanderlukanin13/syntax-next 
Using next() for Python 3 compatibility; ordereddict.py converted from DOS to Unix
 2013-10-22 08:35:09 -07:00
Mikhail Korobov
bf6ef29182 Merge pull request #435 from alexanderlukanin13/syntax-map 
Converted map() to list comprehension/generator for Python3 compatibility
 2013-10-22 07:37:57 -07:00
alexanderlukanin13
a958cb2c5d Using next() for Python 3 compatibility 2013-10-22 13:54:22 +06:00
alexanderlukanin13
6b35166de6 Converted map() to list comprehension or generator for Python 3 compatibility 2013-10-22 13:18:26 +06:00
Pablo Hoffman
911c8082b0 simplified description of crawl command 2013-10-21 14:42:51 -02:00
Pablo Hoffman
e8ee449a2a Merge pull request #432 from darkrho/crawl-url 
Removed URL reference in crawl command and .tld suffix in docs for spider names
 2013-10-21 09:40:58 -07:00
alexanderlukanin13
ae4a61a2d5 Added "from __future__ import print_function" in all relevant places except xlib 2013-10-20 11:55:07 +06:00
Rolando Espinoza La fuente
34543c2b2e DOCS removed .tld suffix for spider names for the sake of consistency. 2013-10-19 23:03:20 -04:00
Rolando Espinoza La fuente
c2ced9a234 Removed reference to URL in crawl command as it's no longer supported. 2013-10-19 22:57:37 -04:00
alexanderlukanin13
d381a35732 Python 3 compatible syntax: print, except, raise, octal numbers; removed Python 2.2 boolean compatibility code in xlib/pydispatch/dispatcher.py 2013-10-20 01:59:35 +06:00
Rolando Espinoza La fuente
dc666f6e5d PEP8 improvements 
* Space lines around classes/functions.
* Space in inline comments.
* Remove redundant backslash between brackets.
 2013-10-19 15:40:42 -04:00
Mikhail Korobov
62fd5b3213 Merge pull request #430 from alexanderlukanin13/cleanup 
pylint cleanup: unused imports and old-style exceptions
 2013-10-18 16:20:04 -07:00
alexanderlukanin13
105e46205b restored import *, added comment 2013-10-19 00:48:22 +06:00
Daniel Graña
1f17230f17 add testing environment for development versions of w3lib and queuelib 2013-10-18 16:08:27 -02:00
alexanderlukanin13
ddcd571000 pylint cleanup: unused imports and old-style exceptions 2013-10-18 19:23:45 +06:00
Mikhail Korobov
2e8cc281b0 Merge pull request #429 from alexanderlukanin13/bugfix 
wrong variable name
 2013-10-18 02:28:06 -07:00
alexanderlukanin13
192c323a0c wrong variable name 2013-10-18 11:30:48 +06:00
Daniel Graña
757e262267 update requirement for cssselect >= 0.9 2013-10-17 09:33:15 -02:00
Daniel Graña
875b07aef8 fix references to old selector naming in docs 2013-10-17 09:33:15 -02:00
Travis Briggs
3043a5ba37 DownloaderMiddleware docs: Update process_request, proper explanation of IgnoreRequest. 
Also:
* Change terminology to eliminate uses of terms such as "request middleware" to refer to the process_request methods of installed middleware.
* Remove description of "immediate redirection", as it is misleading.

Further changes.
 2013-10-17 00:23:21 +00:00
Mikhail Korobov
086b8a20d4 typo fix in TextResponse docs 2013-10-17 04:50:30 +06:00
Pablo Hoffman
951a9f3f4c Merge pull request #226 from scraperdragon/patch-1 
Parameters to Request() in wrong order
 2013-10-16 13:15:52 -07:00