documentation contains everything you need to know about Scrapy. Getting help Having trouble? We’d like to help! Try the FAQ – it’s got answers to some common questions. Looking for specific information NUMBER is an integer. We’ll use that to construct the regular expression for the links to follow: /tor/\d+. We’ll use XPath [http://www.w3.org/TR/xpath] for selecting the data to extract from the web page start_urls = ['http://www.mininova.org/today'] rules = [Rule(SgmlLinkExtractor(allow=['/tor/\d+']), 'parse_torrent')] def parse_torrent(self, response): x = HtmlXPathSelector(response)

Contents 1 Scrapy Documentation, Release 0.14.4 2 Contents CHAPTER 1 Getting help Having trouble? We’d like to help! • Try the FAQ – it’s got answers to some common questions. • Looking for specific information NUMBER is an integer. We’ll use that to construct the regular expression for the links to follow: /tor/\d+. We’ll use XPath for selecting the data to extract from the web page HTML source. Let’s take one of org'] start_urls = ['http://www.mininova.org/today'] rules = [Rule(SgmlLinkExtractor(allow=['/tor/\d+']), 'parse_torrent')] def parse_torrent(self, response): x = HtmlXPathSelector(response) torrent

documentation contains everything you need to know about Scrapy. Getting help Having trouble? We’d like to help! Try the FAQ – it’s got answers to some common questions. Looking for specific information NUMBER is an integer. We’ll use that to construct the regular expression for the links to follow: /tor/\d+. We’ll use XPath [http://www.w3.org/TR/xpath] for selecting the data to extract from the web page start_urls = ['http://www.mininova.org/today'] rules = [Rule(SgmlLinkExtractor(allow=['/tor/\d+']), 'parse_torrent')] def parse_torrent(self, response): x = HtmlXPathSelector(response)

Contents 1 Scrapy Documentation, Release 0.12.0 2 Contents CHAPTER 1 Getting help Having trouble? We’d like to help! • Try the FAQ – it’s got answers to some common questions. • Looking for specific information NUMBER is an integer. We’ll use that to construct the regular expression for the links to follow: /tor/\d+. We’ll use XPath for selecting the data to extract from the web page HTML source. Let’s take one of org'] start_urls = ['http://www.mininova.org/today'] rules = [Rule(SgmlLinkExtractor(allow=['/tor/\d+']), 'parse_torrent')] def parse_torrent(self, response): x = HtmlXPathSelector(response) torrent

Contents 1 Scrapy Documentation, Release 0.16.5 2 Contents CHAPTER 1 Getting help Having trouble? We’d like to help! • Try the FAQ – it’s got answers to some common questions. • Looking for specific information NUMBER is an integer. We’ll use that to construct the regular expression for the links to follow: /tor/\d+. We’ll use XPath for selecting the data to extract from the web page HTML source. Let’s take one of org'] start_urls = ['http://www.mininova.org/today'] rules = [Rule(SgmlLinkExtractor(allow=['/tor/\d+']), 'parse_torrent')] def parse_torrent(self, response): x = HtmlXPathSelector(response) torrent

documentation contains everything you need to know about Scrapy. Getting help Having trouble? We’d like to help! Try the FAQ – it’s got answers to some common questions. Looking for specific information NUMBER is an integer. We’ll use that to construct the regular expression for the links to follow: /tor/\d+. We’ll use XPath [http://www.w3.org/TR/xpath] for selecting the data to extract from the web page start_urls = ['http://www.mininova.org/today'] rules = [Rule(SgmlLinkExtractor(allow=['/tor/\d+']), 'parse_torrent')] def parse_torrent(self, response): x = HtmlXPathSelector(response)

documentation contains everything you need to know about Scrapy. Getting help Having trouble? We’d like to help! Try the FAQ – it’s got answers to some common questions. Looking for specific information Before you start scraping, you will have to set up a new Scrapy project. Enter a directory where you’d like to store your code and run: scrapy startproject tutorial This will create a tutorial directory $ scrapy genspider -l Available templates: basic crawl csvfeed xmlfeed $ scrapy genspider -d basic import scrapy class $classname(scrapy.Spider): name = "$name" allowed_domains = ["$domain"]

documentation contains everything you need to know about Scrapy. Getting help Having trouble? We’d like to help! Try the FAQ – it’s got answers to some common questions. Looking for specific information be intolerably stupid.\u201d" }, { "author": "Groucho Marx", "text": "\u201cOutside of a dog, a book is man's best friend. Inside of a dog it's too dark to read.\u201d" }, { "author": "Steve "Steve Martin", "text": "\u201cA day without sunshine is like, you know, night.\u201d" }, ...] What just happened? When you ran the command scrapy runspider quotes_spider.py, Scrapy looked for a Spider

documentation contains everything you need to know about Scrapy. Getting help Having trouble? We’d like to help! Try the FAQ – it’s got answers to some common questions. Looking for specific information be intolerably stupid.\u201d" }, { "author": "Groucho Marx", "text": "\u201cOutside of a dog, a book is man's best friend. Inside of a dog it's too dark to read.\u201d" }, { "author": "Steve "Steve Martin", "text": "\u201cA day without sunshine is like, you know, night.\u201d" }, ...] What just happened? When you ran the command scrapy runspider quotes_spider.py, Scrapy looked for a Spider

documentation contains everything you need to know about Scrapy. Getting help Having trouble? We’d like to help! Try the FAQ – it’s got answers to some common questions. Looking for specific information be intolerably stupid.\u201d" }, { "author": "Groucho Marx", "text": "\u201cOutside of a dog, a book is man's best friend. Inside of a dog it's too dark to read.\u201d" }, { "author": "Steve "Steve Martin", "text": "\u201cA day without sunshine is like, you know, night.\u201d" }, ...] What just happened? When you ran the command scrapy runspider quotes_spider.py, Scrapy looked for a Spider