BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-25 05:04:03 +00:00
Code Releases Wiki Activity
6,476 Commits 39 Branches 119 Tags
Commit Graph

6476 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Daniel Graña
8e0b640d2b Merge pull request #2826 from dguo/patch-1 
[MRG+1] Tweak the CSVFeedSpider documentation
 2017-07-24 15:28:15 -03:00
Daniel Graña
45b81693b4 Merge pull request #2769 from stummjr/issue-2766 
[MRG+1] Add verification to check if Request callback is callable
 2017-07-24 15:14:33 -03:00
Daniel Graña
50b2567d0d Merge pull request #2848 from redapple/tox-jessie-ssl 
[WIP] Jessie toxenv: Add cryptography as per https://packages.debian.org/jessie/python-cryptography
 2017-07-24 14:44:41 -03:00
Paul Tremberth
1a18587d41 Jessi toxenv: Add cryptography as per https://packages.debian.org/jessie/python-cryptography 2017-07-24 19:30:08 +02:00
Valdir Stumm Junior
bb0bd691d9 Improve error message when callback is not callable 2017-07-24 11:12:09 -03:00
Danny Guo
26c488970c Fix a typo in the Items documentation 2017-07-18 19:56:51 -05:00
Mikhail Korobov
17fe6d2553 Merge pull request #2828 from cconrad/patch-1 
Spelling mistake
 2017-07-15 12:09:45 +02:00
Claus Conrad
18b96dd82a Spelling mistake 2017-07-15 11:31:09 +02:00
Danny Guo
dedc4a8b8f Tweak the CSVFeedSpider documentation 2017-07-13 22:58:10 -05:00
Mikhail Korobov
15abc0b9e3 Merge pull request #2816 from redapple/dns-cache-disabled 
Fix DNS resolver when DNSCACHE_ENABLED=False
 2017-07-06 15:53:33 +05:00
Paul Tremberth
f0ded6b775 Do not cache DNS responses when cache size is 0 2017-07-04 23:18:15 +02:00
Paul Tremberth
1f08d9a648 Add test for DNS cache disabling 2017-07-04 23:10:19 +02:00
Eugenio Lacuesta
793b2376f8 Populate spider variable when using shell.inspect_response 2017-07-03 11:28:04 -03:00
Mikhail Korobov
dd5ab6c5fe Merge pull request #2793 from lopuhin/pypy2 
Fix PyPy test failures
 2017-07-03 18:36:54 +05:00
Konstantin Lopuhin
b0a9236357 Use environment markers for custom PyPy requirements 2017-06-19 19:16:50 +03:00
Konstantin Lopuhin
5ba8e5adc0 Remove duplicate PyPy toxenv from Travis config 
Thanks for the catch @redapple
 2017-06-19 17:45:28 +03:00
Konstantin Lopuhin
271b3a485c Require pypy build to pass 2017-06-19 16:46:16 +03:00
Konstantin Lopuhin
ea08b95280 Remove Jython gc branch: it's not supported 2017-06-19 16:45:29 +03:00
Konstantin Lopuhin
a8df090071 Fix httpcache leveldb tests: gc.collect after del 
LevelDB does not have "official" close method, so we have
to rely on garbage collection to close it.
 2017-06-15 14:29:28 +03:00
Konstantin Lopuhin
b4eb60e527 Install PyPyDispatcher for PyPy tests 
Using https://github.com/lopuhin/pydispatcher, pypy branch.
This is executed as a separate step to avoid changing
default requirements.txt and setup.py. If just added to "deps"
in tox, this install command will be executed as one command
and PyPyDispatcher will not override PyDispatcher.
 2017-06-15 13:34:58 +03:00
Konstantin Lopuhin
19ca986aa1 Move garbage_collect to scrapy.utils.python 2017-06-15 13:14:31 +03:00
Konstantin Lopuhin
7c67047e77 Fix get_func_args tests under PyPy 
On CPython get_func_args does not work correctly for built-in
methods.
 2017-06-15 13:07:59 +03:00
Konstantin Lopuhin
5abb70c8d7 Fix test_weakkeycache on PyPy: run gc.collect() 
One gc.collect() seems to be enough, but it's more reliable
to run it several times (at most 100), until all objects are collected.
 2017-06-15 13:06:30 +03:00
Konstantin Lopuhin
c3d17659b3 Fix queue serialization test on PyPy 
It is not affected by Twisted bug #7989 and is more permissive
with pickling (especially with protocol=2).
 2017-06-15 13:05:45 +03:00
Konstantin Lopuhin
6014856df5 Fix test_output_processor_error undere PyPy 
For float(u'$10') PyPy includes "u'" in the error message,
and it's more fair to check error message on input we are really
passing.
 2017-06-15 12:57:32 +03:00
Konstantin Lopuhin
5a08cf3b96 Fix test_start_requests_errors for PyPy 
Twisted prints errors in DebugInfo.__del__, but PyPy does not run
gc.collect() on exit:
http://doc.pypy.org/en/latest/cpython_differences.html?highlight=gc.collect#differences-related-to-garbage-collection-strategies
 2017-06-15 12:02:47 +03:00
Pengyu CHEN
f712513ed7 Added doc for scrapy.exceptions.DontCloseSpider. Also fixes inaccurate doc for scrapy.signals.spider_idle. 2017-06-15 11:09:17 +08:00
Pengyu CHEN
b33e0d5a54 Added: Now supporting <link> tags in Response.follow 2017-06-14 17:09:42 +08:00
Mikhail Korobov
5aebdac45d Merge pull request #2781 from crasker/patch-1 
use suggest method instead of DEPRECATED one
 2017-06-14 03:29:45 +05:00
Mikhail Korobov
4e84424f26 Merge pull request #2777 from chuanjin/patch-1 
Update extensions.rst
 2017-06-14 03:28:31 +05:00
Casker
ae679f6499 Create item-pipeline.rst 2017-06-09 16:12:20 +08:00
Paul Tremberth
39ad0d0bdd Fix setting name reference 2017-06-06 10:48:30 +02:00
Paul Tremberth
e7061f7a41 Reformat a bit 2017-06-06 10:47:43 +02:00
Chuan Jin
3f8542eb56 Update extensions.rst 
#2759
 2017-06-05 23:31:37 +02:00
Valdir Stumm Junior
4b6f68b9ee make reqser tests create Request with proper callback/errback 2017-06-05 17:26:52 -03:00
Valdir Stumm Junior
0f6f486769 fix parse command issue with callback as a string 2017-06-05 16:19:00 -03:00
Paul Tremberth
fad6b70d92 Use https:// for readthedocs links 2017-06-01 16:41:52 +02:00
Valdir Stumm Junior
60727dedf6 verify if Request callback is callable 2017-05-31 15:00:38 -03:00
Mikhail Korobov
5e1f7a9ead DOC change "releases" section content 2017-05-30 00:50:32 +05:00
Mikhail Korobov
083880888b DOC fixed rst syntax in DOWNLOAD_FAIL_ON_DATALOSS docs 2017-05-30 00:44:11 +05:00
Paul Tremberth
d847e65aae Merge pull request #2762 from stummjr/add-subreddit-to-docs 
Include references to Scrapy subreddit in the docs
 2017-05-29 20:35:40 +02:00
Valdir Stumm Junior
80b160d0d7 include references to scrapy subreddit in the docs 2017-05-29 14:56:49 -03:00
Kurt Peek
af2963d0eb Update autothrottle.rst 
Added missing bullet point for the AUTOTHROTTLE_TARGET_CONCURRENCY setting.
 2017-05-24 19:59:36 +05:00
Paul Tremberth
5f69ec98f7 Bump version: 1.3.2 → 1.4.0 1.4.0 2017-05-18 23:01:05 +02:00
Paul Tremberth
fc2846d637 Set release date for v1.4.0 2017-05-18 22:59:46 +02:00
Paul Tremberth
79b0d8605c Merge pull request #2630 from scrapy/release-notes-1.4.0 
Release notes for 1.4.0
 2017-05-18 22:57:51 +02:00
Mikhail Korobov
76e5b0f65c DOC 1.4 deprecations and backwards incompatible changes, add recent commits to news. 2017-05-18 22:37:15 +02:00
Mikhail Korobov
edcde7a2cf DOC tweak release notes: promote response.follow, mention logging/stats changes 2017-05-18 22:37:15 +02:00
Paul Tremberth
a3d3cd4cb7 Update with latest merges 2017-05-18 22:37:15 +02:00
Paul Tremberth
432668acf7 Mention implementation of #667 2017-05-18 22:37:15 +02:00