From 2abc9bc901491b24ea1a35058ae2b86e44492c88 Mon Sep 17 00:00:00 2001 From: Valdir Stumm Jr Date: Wed, 6 Jan 2016 10:29:45 -0200 Subject: [PATCH] Update deprecated examples * update the scrapy.org example to deal with the new layout. * replaced slashdot.org by reddit.com, because it seems that slashdot is blocking requests. --- docs/topics/shell.rst | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/docs/topics/shell.rst b/docs/topics/shell.rst index 2b118bfbd..3569cbf37 100644 --- a/docs/topics/shell.rst +++ b/docs/topics/shell.rst @@ -106,10 +106,10 @@ Example of shell session ======================== Here's an example of a typical shell session where we start by scraping the -http://scrapy.org page, and then proceed to scrape the http://slashdot.org -page. Finally, we modify the (Slashdot) request method to POST and re-fetch it -getting a HTTP 405 (method not allowed) error. We end the session by typing -Ctrl-D (in Unix systems) or Ctrl-Z in Windows. +http://scrapy.org page, and then proceed to scrape the http://reddit.com +page. Finally, we modify the (Reddit) request method to POST and re-fetch it +getting an error. We end the session by typing Ctrl-D (in Unix systems) or +Ctrl-Z in Windows. Keep in mind that the data extracted here may not be the same when you try it, as those pages are not static and could have changed by the time you test this. @@ -140,24 +140,24 @@ all start with the ``[s]`` prefix):: After that, we can start playing with the objects:: - >>> response.xpath("//h1/text()").extract()[0] - u'Meet Scrapy' + >>> response.xpath('//title/text()').extract_first() + u'Scrapy | A Fast and Powerful Scraping and Web Crawling Framework' - >>> fetch("http://slashdot.org") + >>> fetch("http://reddit.com") [s] Available Scrapy objects: - [s] crawler + [s] crawler [s] item {} - [s] request - [s] response <200 http://slashdot.org> - [s] settings - [s] spider + [s] request + [s] response <200 https://www.reddit.com/> + [s] settings + [s] spider [s] Useful shortcuts: [s] shelp() Shell help (print this help) [s] fetch(req_or_url) Fetch request (or URL) and update local objects [s] view(response) View response in a browser >>> response.xpath('//title/text()').extract() - [u'Slashdot: News for nerds, stuff that matters'] + [u'reddit: the front page of the internet'] >>> request = request.replace(method="POST")