. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . reference. 6 Chapter 2. First steps Scrapy Documentation, Release 0.14.4 Finally, here’s the spider code: class MininovaSpider(CrawlSpider): name = 'mininova.org' allowed_domains = ['mininova.org'] start_urls official release Download Scrapy from the Download page. Scrapy is distributed in two ways: a source code tarball (for Unix and Mac OS X systems) and a Windows installer (for Windows). If you downloaded the

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . /text()[2] For more information about XPath see the XPath reference. Finally, here’s the spider code: from scrapy.contrib.spiders import CrawlSpider, Rule from scrapy.contrib.linkextractors.sgml import Support for creating spiders based on pre-defined templates, to speed up spider creation and make their code more consistent on large projects. See genspider command for more details. • Extensible stats collection

guide Get Scrapy installed on your computer. Scrapy Tutorial Write your first Scrapy project. Examples Learn more by playing with a pre-made Scrapy project. Basic concepts Command line tool Learn your websites. XPath Selectors Extract the data from web pages. Scrapy shell Test your extraction code in an interactive environment. Item Loaders Populate your items with the extracted data. Item Pipeline information about XPath see the XPath reference [http://www.w3.org/TR/xpath]. Finally, here’s the spider code: class MininovaSpider(CrawlSpider): name = 'mininova.org' allowed_domains = ['mininova.org']

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . reference. 6 Chapter 2. First steps Scrapy Documentation, Release 0.20.2 Finally, here’s the spider code: class MininovaSpider(CrawlSpider): name = ’mininova’ allowed_domains = [’mininova.org’] start_urls Support for creating spiders based on pre-defined templates, to speed up spider creation and make their code more consistent on large projects. See genspider command for more details. • Extensible stats collection

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . walk you through an example of a Scrapy Spider using the simplest way to run a spider. Here’s the code for a spider that scrapes famous quotes from website https://quotes.toscrape.com, following the pagi- scraping, you will have to set up a new Scrapy project. Enter a directory where you’d like to store your code and run: scrapy startproject tutorial This will create a tutorial directory with the following contents:

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . reference. 6 Chapter 2. First steps Scrapy Documentation, Release 0.18.4 Finally, here’s the spider code: class MininovaSpider(CrawlSpider): name = 'mininova.org' allowed_domains = ['mininova.org'] start_urls Support for creating spiders based on pre-defined templates, to speed up spider creation and make their code more consistent on large projects. See genspider command for more details. • Extensible stats collection

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . reference. 6 Chapter 2. First steps Scrapy Documentation, Release 0.16.5 Finally, here’s the spider code: class MininovaSpider(CrawlSpider): name = 'mininova.org' allowed_domains = ['mininova.org'] start_urls Support for creating spiders based on pre-defined templates, to speed up spider creation and make their code more consistent on large projects. See genspider command for more details. • Extensible stats collection

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . walk you through an example of a Scrapy Spider using the simplest way to run a spider. Here’s the code for a spider that scrapes famous quotes from website https://quotes.toscrape.com, following the pagi- scraping, you will have to set up a new Scrapy project. Enter a directory where you’d like to store your code and run: scrapy startproject tutorial This will create a tutorial directory with the following contents:

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . walk you through an example of a Scrapy Spider using the simplest way to run a spider. Here’s the code for a spider that scrapes famous quotes from website https://quotes.toscrape.com, following the pagi- scraping, you will have to set up a new Scrapy project. Enter a directory where you’d like to store your code and run: scrapy startproject tutorial This will create a tutorial directory with the following contents:

your websites. XPath Selectors Extract the data from web pages. Scrapy shell Test your extraction code in an interactive environment. Item Loaders Populate your items with the extracted data. Item Pipeline information about XPath see the XPath reference [http://www.w3.org/TR/xpath]. Finally, here’s the spider code: class MininovaSpider(CrawlSpider): name = 'mininova.org' allowed_domains = ['mininova.org'] from the Download page [http://scrapy.org/download/]. Scrapy is distributed in two ways: a source code tarball (for Unix and Mac OS X systems) and a Windows installer (for Windows). If you downloaded the