* delay is enforced per website, not per spider; * document download_delay attribute (it was previously documented only in FAQ about 999 error codes); * document how CONCURRENT_REQUESTS_PER_IP affects download delays.
default. Also deprecated concurrency and delay settings, in favour of using the standard Scrapy ones.