object constructed with the last response fetched 3.6.3 Example of shell session Here’s an example of a typical shell session where we start by scraping the http://scrapy.org page, and then proceed to request method to POST and re-fetch it getting a HTTP 405 (method not allowed) error. We end the session by typing Ctrl-D (in Unix systems) or Ctrl-Z in Windows. Keep in mind that the data extracted here 6023 >>> engine.stop() Connection closed by foreign host. 4.4.4 Telnet Console signals scrapy.management.telnet.update_telnet_vars(telnet_vars) Sent just before the telnet console is opened. You can

XmlXPathSelector object constructed with the last response fetched Example of shell session Here’s an example of a typical shell session where we start by scraping the http://scrapy.org page, and then proceed to request method to POST and re-fetch it getting a HTTP 405 (method not allowed) error. We end the session by typing Ctrl-D (in Unix systems) or Ctrl-Z in Windows. Keep in mind that the data extracted here localhost 6023 >>> engine.stop() Connection closed by foreign host. Telnet Console signals scrapy.management.telnet.update_telnet_vars(telnet_vars) Sent just before the telnet console is opened. You can

extensions, and pipelines). Wide range of built-in middlewares and extensions for: cookies and session handling HTTP compression HTTP authentication HTTP cache user-agent spoofing robots.txt crawl depth response fetched settings - the current Scrapy settings Example of shell session Here’s an example of a typical shell session where we start by scraping the http://scrapy.org page, and then proceed to request method to POST and re-fetch it getting a HTTP 405 (method not allowed) error. We end the session by typing Ctrl-D (in Unix systems) or Ctrl-Z in Windows. Keep in mind that the data extracted here

extensions, and pipelines). • Wide range of built-in middlewares and extensions for: – cookies and session handling – HTTP compression – HTTP authentication – HTTP cache – user-agent spoofing – robots fetched • settings - the current Scrapy settings 3.7.3 Example of shell session Here’s an example of a typical shell session where we start by scraping the http://scrapy.org page, and then proceed to request method to POST and re-fetch it getting a HTTP 405 (method not allowed) error. We end the session by typing Ctrl-D (in Unix systems) or Ctrl-Z in Windows. Keep in mind that the data extracted here

extensions, and pipelines). Wide range of built-in middlewares and extensions for: cookies and session handling HTTP compression HTTP authentication HTTP cache user-agent spoofing robots.txt crawl depth response fetched settings - the current Scrapy settings Example of shell session Here’s an example of a typical shell session where we start by scraping the http://scrapy.org page, and then proceed to request method to POST and re-fetch it getting a HTTP 405 (method not allowed) error. We end the session by typing Ctrl-D (in Unix systems) or Ctrl-Z in Windows. Keep in mind that the data extracted here

extensions, and pipelines). • Wide range of built-in middlewares and extensions for: – cookies and session handling – HTTP compression – HTTP authentication – HTTP cache – user-agent spoofing – robots Scrapy shell 53 Scrapy Documentation, Release 0.14.4 3.7.3 Example of shell session Here’s an example of a typical shell session where we start by scraping the http://scrapy.org page, and then proceed to request method to POST and re-fetch it getting a HTTP 405 (method not allowed) error. We end the session by typing Ctrl-D (in Unix systems) or Ctrl-Z in Windows. Keep in mind that the data extracted here

extensions, and pipelines). • Wide range of built-in middlewares and extensions for: – cookies and session handling – HTTP compression – HTTP authentication – HTTP cache – user-agent spoofing – robots Basic concepts Scrapy Documentation, Release 0.16.5 3.7.3 Example of shell session Here’s an example of a typical shell session where we start by scraping the http://scrapy.org page, and then proceed to request method to POST and re-fetch it getting a HTTP 405 (method not allowed) error. We end the session by typing Ctrl-D (in Unix systems) or Ctrl-Z in Windows. Keep in mind that the data extracted here

extensions, and pipelines). • Wide range of built-in middlewares and extensions for: – cookies and session handling – HTTP compression – HTTP authentication – HTTP cache – user-agent spoofing – robots fetched • settings - the current Scrapy settings 3.7.3 Example of shell session Here’s an example of a typical shell session where we start by scraping the http://scrapy.org page, and then proceed to request method to POST and re-fetch it getting a HTTP 405 (method not allowed) error. We end the session by typing Ctrl-D (in Unix systems) or Ctrl-Z in Windows. Keep in mind that the data extracted here

extensions, and pipelines). • Wide range of built-in middlewares and extensions for: – cookies and session handling – HTTP compression – HTTP authentication – HTTP cache – user-agent spoofing – robots fetched • settings - the current Scrapy settings 3.6.3 Example of shell session Here’s an example of a typical shell session where we start by scraping the http://scrapy.org page, and then proceed to request method to POST and re-fetch it getting a HTTP 405 (method not allowed) error. We end the session by typing Ctrl-D (in Unix systems) or Ctrl-Z in Windows. Keep in mind that the data extracted here

extensions, and pipelines). • Wide range of built-in middlewares and extensions for: – cookies and session handling – HTTP compression – HTTP authentication – HTTP cache – user-agent spoofing – robots Scrapy shell 55 Scrapy Documentation, Release 0.20.2 3.7.3 Example of shell session Here’s an example of a typical shell session where we start by scraping the http://scrapy.org page, and then proceed to request method to POST and re-fetch it getting a HTTP 405 (method not allowed) error. We end the session by typing Ctrl-D (in Unix systems) or Ctrl-Z in Windows. Keep in mind that the data extracted here