1
0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-24 21:24:19 +00:00

6637 Commits

Author SHA1 Message Date
Kurt Peek
af2963d0eb Update autothrottle.rst
Added missing bullet point for the AUTOTHROTTLE_TARGET_CONCURRENCY setting.
2017-05-24 19:59:36 +05:00
Paul Tremberth
5f69ec98f7 Bump version: 1.3.2 → 1.4.0 1.4.0 2017-05-18 23:01:05 +02:00
Paul Tremberth
fc2846d637 Set release date for v1.4.0 2017-05-18 22:59:46 +02:00
Paul Tremberth
79b0d8605c Merge pull request #2630 from scrapy/release-notes-1.4.0
Release notes for 1.4.0
2017-05-18 22:57:51 +02:00
Mikhail Korobov
76e5b0f65c DOC 1.4 deprecations and backwards incompatible changes, add recent commits to news. 2017-05-18 22:37:15 +02:00
Mikhail Korobov
edcde7a2cf DOC tweak release notes: promote response.follow, mention logging/stats changes 2017-05-18 22:37:15 +02:00
Paul Tremberth
a3d3cd4cb7 Update with latest merges 2017-05-18 22:37:15 +02:00
Paul Tremberth
432668acf7 Mention implementation of #667 2017-05-18 22:37:15 +02:00
Paul Tremberth
896c30a8eb Reference items pretty-printing issue number 2017-05-18 22:37:15 +02:00
Paul Tremberth
55d1082360 Reference recent fixes and commits 2017-05-18 22:37:15 +02:00
Paul Tremberth
7d72394794 Reword mention of new response.follow() shortcut 2017-05-18 22:37:15 +02:00
Paul Tremberth
c6464cc4f5 Add verbose introduction to new features 2017-05-18 22:37:14 +02:00
Paul Tremberth
e139d990fc Fix sphinx-build warning on deprecated latex_paper_size 2017-05-18 22:37:14 +02:00
Paul Tremberth
cba55cd190 Rephrase other sections 2017-05-18 22:37:14 +02:00
Paul Tremberth
8729a91f7a Rephrase "New features" section 2017-05-18 22:37:14 +02:00
Paul Tremberth
851adcedf2 List merged pull requests since 1.3.3 2017-05-18 22:37:14 +02:00
Paul Tremberth
0c237c6cac Merge pull request #2750 from scrapy/codecov
tweak coverage reports
2017-05-18 21:38:54 +02:00
Daniel Graña
25f609e2a3 Merge pull request #2675 from simongartz/png-p-to-jpg-conversion-fix
[MRG+1] Fixes conversion of transparent PNG with palette images to jpg #2452
2017-05-18 16:34:02 -03:00
Paul Tremberth
8aa2e4f997 Merge pull request #1829 from nyov/nyov/editor
[MRG+1] Remove dependency on os.environ from default settings
2017-05-18 21:32:51 +02:00
Daniel Graña
b9d3b447a5 Merge pull request #2670 from qhuang872/master
[MRG+1] Update spiders.rst
2017-05-18 16:31:01 -03:00
Paul Tremberth
143810410f Merge pull request #2740 from luzfcb/luzfcb-run-scrapy-from-module
[MRG+1] Add support for executing scrapy using -m option of python
2017-05-18 21:30:19 +02:00
Mikhail Korobov
9ce03d096d codecov config: disable project check, tweak PR comments 2017-05-19 00:01:27 +05:00
Bernardas
1a452c038c increase ptpython priority since it can use other shells as backend 2017-05-18 16:57:13 +00:00
Mikhail Korobov
532400f993 Merge pull request #2643 from harshasrinivas/set-retry-times-per-request
[MRG+1] Add feature to set RETRY_TIMES per request (#2642)
2017-05-17 15:46:33 +05:00
Mikhail Korobov
1acacab93b Merge pull request #2741 from eliat123/2576_cleanup_MEMUSAGE_REPORT
[FIX #2576] cleanup: removed unused MEMUSAGE_REPORT
2017-05-17 13:48:00 +05:00
Daniel Graña
73668ce407 Merge pull request #2721 from HarrisonGregg/feature-drop-from-response-field
[MRG+1] Allow dropping field in from_response formdata
2017-05-16 09:47:45 -03:00
Eli Atzaba
b74b98fa3e cleanup: removed unused MEMUSAGE_REPORT
Signed-off-by: Eli Atzaba <eliat123@gmail.com>
2017-05-16 13:59:58 +03:00
Fábio C. Barrionuevo da Luz
df7a5c4aa4 Add support for executing scrapy using -m option of python
python -m scrapy
2017-05-15 22:52:23 -03:00
Harrison Gregg
ffef828a8d Add test for dropping fields in from_response request body 2017-05-15 12:25:32 -04:00
Harrison Gregg
45a323024c Add documentation for dropping fields in from_response request body 2017-05-15 12:25:32 -04:00
Harrison Gregg
26f723e4e6 Allow formdata value to be None to drop field generated from response 2017-05-15 12:25:27 -04:00
Paul Tremberth
dfe6d3d59a Merge pull request #2456 from elacuesta/feed_export_beautify
[MRG+1] Feed exports: beautify JSON and XML
2017-05-12 18:12:48 +02:00
Paul Tremberth
3a0a86ed31 Clarify FEED_EXPORT_INDENT section 2017-05-12 17:26:17 +02:00
Eugenio Lacuesta
25535dba9c Feed exports: edit note, fix typos 2017-05-10 16:45:15 -03:00
Kurt Peek
548a432951 Minor grammatical changes 2017-05-10 23:48:01 +05:00
Eugenio Lacuesta
63b8caf5de Feed exports: rewrite indentation test without .strip() 2017-05-09 11:58:53 -03:00
Mikhail Korobov
f0122e2cd9 Merge pull request #2729 from yandongxu/master
Fix doc: open file with "wb" mode will get an error in python 3
2017-05-08 16:24:18 +05:00
yandongxu
4966dd7a7f Fix doc: open file with "wb" mode will get an error in python 3 2017-05-08 18:50:30 +08:00
Liu Siyuan
6c1cacb5d5 [MRG+1] doc: fix documentation error in link-extractor.rst (#2676)
* fix doc error in link-extractor.rst

* remove the import clause

* update based on suggestion
2017-05-06 02:47:06 +05:00
Daniel Graña
362d6f2d20 Merge pull request #2622 from rolando-contrib/download-maxsize-abort
[MRG+1] Abort connection earlier and avoid to buffer data when max size limit is reached
2017-05-04 11:44:51 -03:00
Mikhail Korobov
7fc11c1348 Merge pull request #2720 from redapple/update-test-server-cert
Change "localhost" test server certificate
2017-04-28 02:58:46 +05:00
Paul Tremberth
6d14e392f1 Remove old test certificate+key 2017-04-27 23:35:01 +02:00
Paul Tremberth
e6ab8bc9a5 Change "localhost" test server certificate 2017-04-27 23:22:25 +02:00
Rolando Espinoza
2b34c6edff Abort connection earlier and avoid to buffer data
A symptom of this issue was having the log message "Received (X) bytes
larger than download max size (Y)" several times printed, with increased
X values.
2017-04-27 13:28:16 -03:00
Daniel Graña
2d66c58e01 Merge pull request #2678 from redapple/double-content-length-0-post
Set bodyproducer with empty content for POST
2017-04-26 16:10:21 -03:00
Martín Gaitán
4bc0c6b0f4 Update practices.rst
fix a typo
2017-04-26 14:12:57 +05:00
Paul Tremberth
c3d0f9b6c1 Add test for non-duplicated Content-Length: 0 for bodyless POST 2017-04-25 17:03:41 +02:00
Paul Tremberth
a63d9f502f Restore comments on why POST needs Content-Length: 0 2017-04-25 17:03:29 +02:00
Paul Tremberth
b1a0a6e258 Make mockserver runnable outside of tox
Add POST support for Echo resource
2017-04-25 17:01:54 +02:00
Paul Tremberth
97fc68fa16 Refactor conditions on body producer 2017-04-24 22:08:39 +02:00