BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-25 12:03:58 +00:00
Code Releases Wiki Activity

added comment downloader backout policy

Browse Source
This commit is contained in:
Pablo Hoffman 2009-09-04 01:16:58 -03:00
parent 2ae11e9220
commit dad05957f3
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
scrapy/core/downloader/manager.py
View File

@ -42,6 +42,7 @@ class SiteInfo(object):
return self.max_concurrent_requests - len(self.transferring)
def needs_backout(self):
# use self.active to include requests in the downloader middleware
return len(self.active) > 2 * self.max_concurrent_requests