BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-27 00:43:48 +00:00
Code Releases Wiki Activity
5,464 Commits 39 Branches 119 Tags
Commit Graph

5464 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Elias Dorneles
aaccdbb49b Bump version: 1.1.0rc1 → 1.2.0dev1 2016-02-03 21:54:39 -02:00
Elias Dorneles
a0b63a7707 Bump version: 1.1.0dev1 → 1.1.0rc1 2016-02-03 21:52:47 -02:00
Elias Dorneles
94b2c47da5 Merge pull request #1751 from scrapy/release-notes-1.1 
[MRG+1] Release notes 1.1
 2016-02-03 21:51:10 -02:00
Elias Dorneles
ca08314654 makes more explicit source of problems in Python 3 limitations 2016-02-03 21:38:25 -02:00
Elias Dorneles
2167573b32 fix comment about disabling robots.txt 2016-02-03 21:31:27 -02:00
Elias Dorneles
3180abc800 applying more review comments 2016-02-03 21:28:05 -02:00
Elias Dorneles
d40add7b27 add note about robots.txt waiting and make it explicit builtin extensions only are ported 2016-02-03 21:18:25 -02:00
Elias Dorneles
9cfefd5277 favor beta over basic to describe PY3 support 2016-02-03 20:59:56 -02:00
Elias Dorneles
2f697ce59f added note about how to revert to old behavior for robotstxt 2016-02-03 20:51:26 -02:00
Elias Dorneles
89a088b3c3 applying review comments 2016-02-03 20:45:55 -02:00
Daniel Graña
a214b131fc Merge pull request #1748 from scrapy/py3-crawlerrunner-stop 
[MRG+1] PY3 fixed CrawlerRunner.stop
 2016-02-03 19:41:43 -03:00
Paul Tremberth
972f192568 Blank line before CoC 2016-02-03 22:58:49 +01:00
Paul Tremberth
728e505e53 Merge remote-tracking branch 'origin/master' into release-notes-1.1 2016-02-03 22:47:15 +01:00
Paul Tremberth
725a209e83 Merge pull request #1755 from stummjr/add-overview-to-release-notes 
Add overview to release notes and other edits
 2016-02-03 22:37:02 +01:00
Valdir Stumm Jr
dd7b0eb359 Updates to release notes 2016-02-03 19:23:02 -02:00
Paul Tremberth
51dc741fa2 Add 1.0.5 release notes 2016-02-03 21:07:21 +01:00
Valdir Stumm Jr
0f816524e8 edit API changes, Deprecations, and Relocations sections. 2016-02-03 16:21:32 -02:00
Paul Tremberth
fd3193da86 Reword some of "New Features and Enhancements" 2016-02-03 18:19:38 +01:00
Paul Tremberth
3b2a6abc0b Format "Code refactoring" subsection 2016-02-03 17:37:58 +01:00
Paul Tremberth
ed02455010 Format "Logging" subsection 2016-02-03 17:27:43 +01:00
Elias Dorneles
a8a6f050e7 Merge pull request #1735 from ArturGaspar/master 
[MRG+1] Fix for KeyError in robots.txt middleware
 2016-02-03 13:30:06 -02:00
Paul Tremberth
e4cd95444c Merge pull request #1752 from stummjr/release-notes-1.1 
Release notes: edits on "Py3 Support" and "New features" sections
 2016-02-03 16:17:59 +01:00
Mikhail Korobov
43a53aca12 Merge pull request #1746 from redapple/shell-settings-logging 
[MRG+1] Remove __str__ and __repr__ from settings, introduce copy_to_dict()
 2016-02-03 19:59:24 +05:00
Valdir Stumm Jr
1fd95c9c9f fixed references to issues and fit text to 80 cols in the first two sections 2016-02-03 12:45:48 -02:00
Valdir Stumm Jr
a21c90bee7 edits on Py3 Support and New features sections 2016-02-03 11:54:46 -02:00
Paul Tremberth
44d8df2060 Add versionadded note for MEMUSAGE_CHECK_INTERVAL_SECONDS 2016-02-03 12:35:26 +01:00
Paul Tremberth
2b033eebce Fix recently added HTTPCACHE_ settings versionadded notes 2016-02-03 12:34:44 +01:00
Paul Tremberth
142aa21737 Add AUTOTHROTTLE_TARGET_CONCURRENCY versionadded note 2016-02-03 12:33:10 +01:00
Paul Tremberth
db0697bc06 Add 1.1 release notes (draft) 2016-02-03 12:32:40 +01:00
Mikhail Korobov
65fb67f2db PY3 fixed CrawlerRunner.stop 2016-02-03 01:01:16 +05:00
Nicolás Alejandro Ramírez Quiros
8a391a557f Merge pull request #1741 from scrapy/py3-downloader-gc 
[MRG+1] PY3 fix downloader slots GC
 2016-02-02 12:17:52 -03:00
Julia Medina
f2668c316f Merge pull request #1747 from stummjr/fix-xml-exporter-for-nonstring-types 
[MRG+1] Fix bug on XMLItemExporter with non-string fields in items
 2016-02-02 01:35:39 -03:00
Paul Tremberth
268e912273 Add pretty-printting of settings as dict if using IPython shell 
Suggested by @digenis
see http://ipython.readthedocs.org/en/stable/api/generated/IPython.lib.pretty.html?#extending
 2016-02-01 12:43:27 +01:00
stummjr
bb2cf7c0d7 Fixed bug on XMLItemExporter with non-string fields in items 2016-01-30 10:00:06 -02:00
Mikhail Korobov
f9dc02e23a PY3 fix downloader slots GC 2016-01-30 04:37:22 +05:00
Paul Tremberth
d843a0aae8 Amend "settings" command to output JSON for dict settings 2016-01-29 21:12:03 +01:00
Paul Tremberth
aa78758bc7 Update tests for settings copy_to_dict() 2016-01-29 18:59:12 +01:00
Paul Tremberth
a1ebff83d3 Remove __str__ and __repr__ from settings, introduce copy_to_dict() instead 
Settings instances as dict's are easier to print or pretty print in the shell

Fixes #1732
 2016-01-29 18:39:34 +01:00
Mikhail Korobov
a35aec71e9 Merge pull request #1710 from redapple/1550-shell_file-cont 
[MRG+1] shell command's ability to open local files + tests
 2016-01-29 21:03:40 +05:00
Paul Tremberth
78f00401cd Remove unused import in tests 2016-01-29 16:56:05 +01:00
Mikhail Korobov
505e87258c Merge pull request #1742 from stummjr/add-tests-to-exporters 
[MRG+1] Include tests for non-string items to Exporters
 2016-01-29 03:27:38 +05:00
stummjr
cf2ebb0687 Include tests for exporters: JSON, JSON-Lines, Pickle and Marshal. 2016-01-28 19:43:53 -02:00
Paul Tremberth
e9f6b98816 Amend guess_scheme() docstring 2016-01-28 16:26:45 +01:00
Paul Tremberth
c6f374f2eb Merge remote-tracking branch 'origin/master' into 1550-shell_file-cont 2016-01-28 14:02:48 +01:00
Paul Tremberth
481e251775 Move guess_scheme() tests to relevant test module 2016-01-28 13:51:50 +01:00
Paul Tremberth
cae268402d Move guess_scheme() to scrapy.utils.url 2016-01-28 13:42:04 +01:00
Artur Gaspar
90e3ae1c58 Do not forget failed requests in robots.txt middleware. 2016-01-27 21:00:35 -02:00
Mikhail Korobov
4c9a6ef3ce Merge pull request #1740 from scrapy/response-text-micro-optimize 
[MRG+1] micro-optimize response.text
 2016-01-28 01:39:48 +05:00
Mikhail Korobov
3e080c3c52 call .text from .body_as_unicode() and not the other way around 2016-01-28 00:59:27 +05:00
Mikhail Korobov
394b991e3a Merge pull request #1737 from stummjr/fix-py3-exporters-multiple-types 
[MRG+1] Fix PythonItemExporter and CSVExporter for non-string item types
 2016-01-28 00:46:00 +05:00