BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-22 23:24:11 +00:00
Code Releases Wiki Activity
6,401 Commits 39 Branches 119 Tags
Commit Graph

6401 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Konstantin Lopuhin
271b3a485c Require pypy build to pass 2017-06-19 16:46:16 +03:00
Konstantin Lopuhin
ea08b95280 Remove Jython gc branch: it's not supported 2017-06-19 16:45:29 +03:00
Konstantin Lopuhin
a8df090071 Fix httpcache leveldb tests: gc.collect after del 
LevelDB does not have "official" close method, so we have
to rely on garbage collection to close it.
 2017-06-15 14:29:28 +03:00
Konstantin Lopuhin
b4eb60e527 Install PyPyDispatcher for PyPy tests 
Using https://github.com/lopuhin/pydispatcher, pypy branch.
This is executed as a separate step to avoid changing
default requirements.txt and setup.py. If just added to "deps"
in tox, this install command will be executed as one command
and PyPyDispatcher will not override PyDispatcher.
 2017-06-15 13:34:58 +03:00
Konstantin Lopuhin
19ca986aa1 Move garbage_collect to scrapy.utils.python 2017-06-15 13:14:31 +03:00
Konstantin Lopuhin
7c67047e77 Fix get_func_args tests under PyPy 
On CPython get_func_args does not work correctly for built-in
methods.
 2017-06-15 13:07:59 +03:00
Konstantin Lopuhin
5abb70c8d7 Fix test_weakkeycache on PyPy: run gc.collect() 
One gc.collect() seems to be enough, but it's more reliable
to run it several times (at most 100), until all objects are collected.
 2017-06-15 13:06:30 +03:00
Konstantin Lopuhin
c3d17659b3 Fix queue serialization test on PyPy 
It is not affected by Twisted bug #7989 and is more permissive
with pickling (especially with protocol=2).
 2017-06-15 13:05:45 +03:00
Konstantin Lopuhin
6014856df5 Fix test_output_processor_error undere PyPy 
For float(u'$10') PyPy includes "u'" in the error message,
and it's more fair to check error message on input we are really
passing.
 2017-06-15 12:57:32 +03:00
Konstantin Lopuhin
5a08cf3b96 Fix test_start_requests_errors for PyPy 
Twisted prints errors in DebugInfo.__del__, but PyPy does not run
gc.collect() on exit:
http://doc.pypy.org/en/latest/cpython_differences.html?highlight=gc.collect#differences-related-to-garbage-collection-strategies
 2017-06-15 12:02:47 +03:00
Mikhail Korobov
5aebdac45d Merge pull request #2781 from crasker/patch-1 
use suggest method instead of DEPRECATED one
 2017-06-14 03:29:45 +05:00
Mikhail Korobov
4e84424f26 Merge pull request #2777 from chuanjin/patch-1 
Update extensions.rst
 2017-06-14 03:28:31 +05:00
Casker
ae679f6499 Create item-pipeline.rst 2017-06-09 16:12:20 +08:00
Paul Tremberth
39ad0d0bdd Fix setting name reference 2017-06-06 10:48:30 +02:00
Paul Tremberth
e7061f7a41 Reformat a bit 2017-06-06 10:47:43 +02:00
Chuan Jin
3f8542eb56 Update extensions.rst 
#2759
 2017-06-05 23:31:37 +02:00
Paul Tremberth
fad6b70d92 Use https:// for readthedocs links 2017-06-01 16:41:52 +02:00
Paul Tremberth
d847e65aae Merge pull request #2762 from stummjr/add-subreddit-to-docs 
Include references to Scrapy subreddit in the docs
 2017-05-29 20:35:40 +02:00
Valdir Stumm Junior
80b160d0d7 include references to scrapy subreddit in the docs 2017-05-29 14:56:49 -03:00
Kurt Peek
af2963d0eb Update autothrottle.rst 
Added missing bullet point for the AUTOTHROTTLE_TARGET_CONCURRENCY setting.
 2017-05-24 19:59:36 +05:00
Paul Tremberth
5f69ec98f7 Bump version: 1.3.2 → 1.4.0 1.4.0 2017-05-18 23:01:05 +02:00
Paul Tremberth
fc2846d637 Set release date for v1.4.0 2017-05-18 22:59:46 +02:00
Paul Tremberth
79b0d8605c Merge pull request #2630 from scrapy/release-notes-1.4.0 
Release notes for 1.4.0
 2017-05-18 22:57:51 +02:00
Mikhail Korobov
76e5b0f65c DOC 1.4 deprecations and backwards incompatible changes, add recent commits to news. 2017-05-18 22:37:15 +02:00
Mikhail Korobov
edcde7a2cf DOC tweak release notes: promote response.follow, mention logging/stats changes 2017-05-18 22:37:15 +02:00
Paul Tremberth
a3d3cd4cb7 Update with latest merges 2017-05-18 22:37:15 +02:00
Paul Tremberth
432668acf7 Mention implementation of #667 2017-05-18 22:37:15 +02:00
Paul Tremberth
896c30a8eb Reference items pretty-printing issue number 2017-05-18 22:37:15 +02:00
Paul Tremberth
55d1082360 Reference recent fixes and commits 2017-05-18 22:37:15 +02:00
Paul Tremberth
7d72394794 Reword mention of new response.follow() shortcut 2017-05-18 22:37:15 +02:00
Paul Tremberth
c6464cc4f5 Add verbose introduction to new features 2017-05-18 22:37:14 +02:00
Paul Tremberth
e139d990fc Fix sphinx-build warning on deprecated latex_paper_size 2017-05-18 22:37:14 +02:00
Paul Tremberth
cba55cd190 Rephrase other sections 2017-05-18 22:37:14 +02:00
Paul Tremberth
8729a91f7a Rephrase "New features" section 2017-05-18 22:37:14 +02:00
Paul Tremberth
851adcedf2 List merged pull requests since 1.3.3 2017-05-18 22:37:14 +02:00
Paul Tremberth
0c237c6cac Merge pull request #2750 from scrapy/codecov 
tweak coverage reports
 2017-05-18 21:38:54 +02:00
Daniel Graña
25f609e2a3 Merge pull request #2675 from simongartz/png-p-to-jpg-conversion-fix 
[MRG+1] Fixes conversion of transparent PNG with palette images to jpg #2452
 2017-05-18 16:34:02 -03:00
Paul Tremberth
8aa2e4f997 Merge pull request #1829 from nyov/nyov/editor 
[MRG+1] Remove dependency on os.environ from default settings
 2017-05-18 21:32:51 +02:00
Daniel Graña
b9d3b447a5 Merge pull request #2670 from qhuang872/master 
[MRG+1] Update spiders.rst
 2017-05-18 16:31:01 -03:00
Paul Tremberth
143810410f Merge pull request #2740 from luzfcb/luzfcb-run-scrapy-from-module 
[MRG+1] Add support for executing scrapy using -m option of python
 2017-05-18 21:30:19 +02:00
Mikhail Korobov
9ce03d096d codecov config: disable project check, tweak PR comments 2017-05-19 00:01:27 +05:00
Mikhail Korobov
532400f993 Merge pull request #2643 from harshasrinivas/set-retry-times-per-request 
[MRG+1] Add feature to set RETRY_TIMES per request (#2642)
 2017-05-17 15:46:33 +05:00
Mikhail Korobov
1acacab93b Merge pull request #2741 from eliat123/2576_cleanup_MEMUSAGE_REPORT 
[FIX #2576] cleanup: removed unused MEMUSAGE_REPORT
 2017-05-17 13:48:00 +05:00
Daniel Graña
73668ce407 Merge pull request #2721 from HarrisonGregg/feature-drop-from-response-field 
[MRG+1] Allow dropping field in from_response formdata
 2017-05-16 09:47:45 -03:00
Eli Atzaba
b74b98fa3e cleanup: removed unused MEMUSAGE_REPORT 
Signed-off-by: Eli Atzaba <eliat123@gmail.com>
 2017-05-16 13:59:58 +03:00
Fábio C. Barrionuevo da Luz
df7a5c4aa4 Add support for executing scrapy using -m option of python 
python -m scrapy
 2017-05-15 22:52:23 -03:00
Harrison Gregg
ffef828a8d Add test for dropping fields in from_response request body 2017-05-15 12:25:32 -04:00
Harrison Gregg
45a323024c Add documentation for dropping fields in from_response request body 2017-05-15 12:25:32 -04:00
Harrison Gregg
26f723e4e6 Allow formdata value to be None to drop field generated from response 2017-05-15 12:25:27 -04:00
Paul Tremberth
dfe6d3d59a Merge pull request #2456 from elacuesta/feed_export_beautify 
[MRG+1] Feed exports: beautify JSON and XML
 2017-05-12 18:12:48 +02:00