Firefox for scraping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 5.3 Using Firebug for scraping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . existing cookies, see a list of cookies for the current site, manage cookies permissions and a lot more. 5.3 Using Firebug for scraping 5.3.1 Introduction This document explains how to use Firebug (a Firefox that there are more subcategories than the ones being shown in this page, so we’ll keep looking: 5.3. Using Firebug for scraping 73 Scrapy Documentation, Release 0.9 As expected, the subcategories contain

Firefox for scraping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.3 Using Firebug for scraping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . existing cookies, see a list of cookies for the current site, manage cookies permissions and a lot more. 5.3 Using Firebug for scraping 5.3.1 Introduction This document explains how to use Firebug (a Firefox a very tedious task. In the following screenshot you can see the Inspect Element tool in action. 5.3. Using Firebug for scraping 83 Scrapy Documentation, Release 0.12.0 At first sight, we can see that

Firefox for scraping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5.3 Using Firebug for scraping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . existing cookies, see a list of cookies for the current site, manage cookies permissions and a lot more. 5.3 Using Firebug for scraping Note: Google Directory, the example website used in this guide is no longer that there are more subcategories than the ones being shown in this page, so we’ll keep looking: 5.3. Using Firebug for scraping 85 Scrapy Documentation, Release 0.14.4 As expected, the subcategories

Spiders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 5.3 Spiders Contracts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . received for %s' % response.url, level=log.WARNING) For more information, check the Logging section. 5.3 Spiders Contracts New in version 0.15. Note: This is a new feature (introduced in Scrapy 0.15) and @url http://www.amazon.com/s?field-keywords=selfish+gene @returns items 1 16 @returns requests 0 0 5.3. Spiders Contracts 83 Scrapy Documentation, Release 0.16.5 @scrapes Title Author Year Price """

Spiders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 5.3 Spiders Contracts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . warning('No item received for %s', response.url) For more information, check the Logging section. 5.3 Spiders Contracts New in version 0.15. Testing spiders can get particularly annoying and while nothing parses a sample response. Some contracts are mingled with this docstring. (continues on next page) 5.3. Spiders Contracts 163 Scrapy Documentation, Release 2.2.1 (continued from previous page) @url http://www

Spiders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 5.3 Spiders Contracts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . warning('No item received for %s', response.url) For more information, check the Logging section. 5.3 Spiders Contracts New in version 0.15. Note: This is a new feature (introduced in Scrapy 0.15) and scrapy.contracts import Contract from scrapy.exceptions import ContractFail (continues on next page) 5.3. Spiders Contracts 139 Scrapy Documentation, Release 1.5.2 (continued from previous page) class

Spiders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 5.3 Spiders Contracts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . warning('No item received for %s', response.url) For more information, check the Logging section. 5.3 Spiders Contracts New in version 0.15. Note: This is a new feature (introduced in Scrapy 0.15) and response received. Raise scrapy. exceptions.ContractFail in order to get the failures pretty printed: 5.3. Spiders Contracts 145 Scrapy Documentation, Release 1.6.0 from scrapy.contracts import Contract

Spiders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 5.3 Spiders Contracts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the Logging section. 146 Chapter 5. Solving specific problems Scrapy Documentation, Release 1.7.4 5.3 Spiders Contracts New in version 0.15. Note: This is a new feature (introduced in Scrapy 0.15) and SPIDER_CONTRACTS = { 'myproject.contracts.ResponseCheck': 10, 'myproject.contracts.ItemValidate': 10, } 5.3. Spiders Contracts 147 Scrapy Documentation, Release 1.7.4 Each contract must inherit from Contract

Spiders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 5.3 Spiders Contracts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . warning('No item received for %s', response.url) For more information, check the Logging section. 5.3 Spiders Contracts New in version 0.15. Testing spiders can get particularly annoying and while nothing 0 0 @scrapes Title Author Year Price """ This callback is tested using three built-in contracts: 5.3. Spiders Contracts 159 Scrapy Documentation, Release 1.8.4 class scrapy.contracts.default.UrlContract

Spiders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5.3 Spiders Contracts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . received for %s' % response.url, level=log.WARNING) For more information, check the Logging section. 5.3 Spiders Contracts New in version 0.15. Note: This is a new feature (introduced in Scrapy 0.15) and