BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-24 21:24:19 +00:00
Code Releases Wiki Activity

Added FAQ entry about response code 999

Browse Source
This commit is contained in:
Pablo Hoffman 2010-08-19 16:51:51 -03:00
parent 2ff5a83b7a
commit 3d8151bb26
1 changed files with 17 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
docs/faq.rst
View File

@ -173,3 +173,20 @@ Can I return (Twisted) deferreds from signal handlers?
Some signals support returning deferreds form their handlers, others don't. See
the :ref:`topics-signals-ref` to know which ones.
What does the response status code 999 means?
---------------------------------------------
999 is a custom reponse status code used by Yahoo sites to throttle requests.
Try slowing down the crawling speed by using a download delay in your spider::
class MySpider(CrawlSpider):
name = 'myspider'
download_delay = 1
# [ ... rest of the spider code ... ]
Or by setting a global download delay in your project with the
:setting:`DOWNLOAD_DELAY` setting.