.. _topics-index: Using Scrapy ============ This section describes all key concepts of Scrapy. .. toctree:: :maxdepth: 1 architecture shell spiders selectors link-extractors items item-pipeline downloader-middleware spider-middleware settings extensions stats webconsole telnetconsole robotstxt firefox firebug