BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-25 00:24:31 +00:00
Code Releases Wiki Activity
5,042 Commits 39 Branches 119 Tags
Commit Graph

5042 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yaroslav Halchenko
48582be9a6 DOC(ENH): specify path to rtd theme explicitly 2015-06-25 10:46:22 -04:00
Yaroslav Halchenko
49fe915798 BF: fail if docs failed to build 2015-06-25 10:12:56 -04:00
Pablo Hoffman
8a140b6ba1 Merge pull request #1315 from scrapy/downloader-slots-cleanup 
Small downloader slots cleanup
 2015-06-24 16:25:13 -03:00
Mikhail Korobov
ea4b175c2d Merge pull request #1254 from scrapy/bugfix-1232 
[MRG +1] Unquote request path before passing to FTPClient, it already escape paths
 2015-06-24 18:04:45 +05:00
Mikhail Korobov
08501bff92 Merge pull request #1316 from dallagi/patch-1 
DOC Bring Ubuntu and Archlinux outside of Windows subsection
 2015-06-24 17:38:12 +05:00
Marco DallaG
44c8ef2d47 DOC Bring Ubuntu and Archlinux outside of Windows subsection 
In the installation guide, in the "Platform specific installation notes" section, Ubuntu and ArchLinux are currently subsections of Windows, which does not make sense imho.
This commit changes the section tree from:

Platform specific installation notes
- Windows
  -- Archlinux
  -- Ubuntu

To:

Platform specific installation notes
- Windows
- Ubuntu
- Archlinux
 2015-06-24 11:44:48 +02:00
Mikhail Korobov
cb0445e8b8 Slot.__str__ and __repr__ methods; remove unused import 2015-06-24 03:22:23 +05:00
Mikhail Korobov
8da9836db3 don't access settings from downloader.Slot 2015-06-24 03:20:46 +05:00
Elias Dorneles
eb8fed4ecd support empty password for http_proxy config 2015-06-23 00:43:15 -03:00
Pablo Hoffman
38e5bfb61c remove version suffix from ubuntu package 2015-06-22 10:57:24 -03:00
Julia Medina
af97c373ea Update release date for 1.0 
(cherry picked from commit c89fa29c415f669fd42ad4bae88e754ed0194a4f)
 2015-06-19 18:53:54 -03:00
Daniel Graña
47b1b748c2 Generated version as pep440 and dpkg compatible 2015-06-16 00:16:09 +00:00
Daniel Graña
ec56ed5870 Merge pull request #1307 from agusc/master 
removed SUFFIX from scrapy package name
 2015-06-15 18:02:14 -03:00
agusc
140f58251b removed SUFFIX from scrapy name package 2015-06-15 17:02:09 -03:00
Elias Dorneles
2de5c66058 improvements for docs on how to access settings 2015-06-15 13:07:55 -03:00
Julia Medina
f958fb9d62 Update release notes for 1.0.0rc3 2015-06-12 16:42:39 -03:00
Julia Medina
fa1c25c840 Merge pull request #1286 from scrapy/configure_logging 
configure_logging: change the meaning of settings=None
 2015-06-12 13:22:42 -03:00
Julia Medina
36bc912cdd DOC indent additional docs for configure_logging 2015-06-12 13:00:31 -03:00
Daniel Graña
d862f5acff Merge pull request #1297 from darkrho/leveldb-compact 
Do leveldb compactation for httpcache on closing.
 2015-06-12 11:16:16 -03:00
Mikhail Korobov
aaaba9f34d Merge pull request #1298 from bcrowe/fix-typos 
Fix a couple typos
 2015-06-12 16:02:53 +05:00
Bryan Crowe
6a4c475e87 Fix a couple typos 2015-06-11 19:47:30 -04:00
Rolando Espinoza
877c25793a Do leveldb compactation for the httpcache on closing. 2015-06-11 13:27:49 -04:00
Daniel Graña
5bd0395be4 Merge pull request #1291 from scrapy/signalmanager-docstrings 
DOC SignalManager docstrings. See GH-713.
 2015-06-10 16:28:35 -03:00
Daniel Graña
6446652635 Merge pull request #1294 from berkerpeksag/ignore-warnings 
Ignore ScrapyDeprecationWarning warnings properly.
 2015-06-10 16:07:21 -03:00
Mikhail Korobov
6c9daf3a95 DOC remove unnecessary links; fix references in send_catch_log_deferred docstring 2015-06-10 01:44:19 +05:00
Mikhail Korobov
a611f8dd2d DOC remove FailureFormatter mentions, stop copy-pasting configure_logging docstring 2015-06-09 22:57:18 +05:00
Berker Peksag
8a48d9c6a8 Ignore ScrapyDeprecationWarning warnings properly. 
Conflicts:

	tests/test_utils_deprecate.py
 2015-06-09 18:22:03 +03:00
Daniel Graña
90aa5c075d Merge pull request #1284 from scrapy/crawler-cleanup 
CrawlerProcess cleanup
 2015-06-09 11:21:24 -03:00
Julia Medina
f91461c256 Merge pull request #1292 from scrapy/spidererror-docs 
spider_error doesn't support deferreds
 2015-06-09 11:19:10 -03:00
Julia Medina
9ca87c5b6b Merge pull request #1293 from berkerpeksag/fix-typos 
Fix typos in scrapy/commands/setting.py
 2015-06-09 11:08:10 -03:00
Berker Peksag
66e5591eb9 Fix typos in scrapy/commands/setting.py 
* intepreted -> interpreted
* "a list" instead of "an float" in --getlist help
* "an float"-> "a float"

Also, backslashes were redundant. So I removed them.
 2015-06-09 04:29:24 +03:00
Mikhail Korobov
790c67b643 DOC spider_error doesn't support deferreds 2015-06-09 02:20:10 +05:00
Mikhail Korobov
1740fcf1a6 DOC SignalManager docstrings. See GH-713. 
This change is not 100% backwards compatible because of *args changes.
Their usage was not documented, so we're not breaking public interface.
 2015-06-08 21:05:58 +05:00
Mikhail Korobov
9a787893e3 (backwards-incompatible) allow to pass settings=None to configure_logging 
* use explicit argument for disabling root handler;
* handle LOG_STDOUT even if install_root_handler is False
 2015-06-08 19:54:18 +05:00
Mikhail Korobov
3cbf8a0b2b extract CrawlerRunner._crawl method which always expects Crawler 
It provides an extension point where crawler instance is available;
it should make it easier to write alternative CrawlerRunner.crawl
implementations.

See also: https://github.com/scrapy/scrapy/pull/1256
 2015-06-08 18:35:44 +05:00
Daniel Graña
87293965db Merge pull request #1285 from scrapy/optional-settings-arguments 
make it easier to use default settings
 2015-06-07 20:29:24 -03:00
Chris Nilsson
61dec83f70 Moved default value of MEMUSAGE_CHECK_INTERVAL_SECONDS to default_settings 2015-06-06 11:19:29 +10:00
Chris Nilsson
0c532baf4c Removed typo, and clarified time unit of setting 2015-06-06 11:18:13 +10:00
Mikhail Korobov
d047665c02 make "settings" argument optional for Crawler, CrawlerRunner and CrawlerProcess 2015-06-06 03:23:13 +05:00
Mikhail Korobov
64399d18d8 Stop reactor on Ctrl-C regardless of 'stop_after_crawl'. Fixes GH-1279. 2015-06-06 02:53:36 +05:00
Mikhail Korobov
33d145e2f5 CrawlerProcess cleanup 
* remove unneeded lambda;
* extract _get_dns_resolver method and format code to pep8.
 2015-06-06 02:49:39 +05:00
Julia Medina
24d8a85269 Update release notes for 1.0.0rc2 
(cherry picked from commit 6e61d54168cf471363be3e7e54d75ad544b9f6e1)
 2015-06-05 17:11:40 -03:00
Chris Nilsson
eae25a04d9 Added MEMUSAGE_CHECK_INTERVAL_SECONDS to Memory usage extension options. 
Kept the default as it was, at 60.0 seconds. But added a setting to
allow this to be changed as desired.
 2015-06-06 00:39:14 +10:00
Daniel Graña
d9bcd48606 Merge pull request #1278 from Curita/remove-tz-aware-logformat 
Remove deprecated %z formatting from the default LOG_DATEFORMAT
 2015-06-04 13:39:01 -03:00
Julia Medina
367ea81e71 Remove deprecated %z formatting from the default LOG_DATEFORMAT 2015-06-04 04:11:23 +08:00
Mikhail Korobov
f312ffcb54 Merge pull request #1276 from scrapy/fix-spider-settings 
Fix Spider.custom_settings
 2015-06-03 22:14:04 +05:00
Mikhail Korobov
d42c420a6d fixed spider custom_settings 
https://github.com/scrapy/scrapy/pull/1128 moved spidercls.update_settings
call to a later stage; this commit moves it back.
 2015-06-03 04:29:10 +05:00
Mikhail Korobov
cc2f3e1b46 TST a test case to show custom_settings doesn't always work 2015-06-03 04:26:20 +05:00
Daniel Graña
d52cf8bb03 Merge pull request #1267 from Curita/fix-1265 
Fix #1265
 2015-06-01 20:31:46 -03:00
Julia Medina
ffc7b7fd6c Add helper to update deprecated class paths 2015-06-01 17:01:33 -03:00