options: • --callback or -c: spider method to use as callback for parsing the response • --rules or -r: use CrawlSpider rules to discover the callback (ie. spider method) to use for parsing the response HtmlXPathSelector(response) item = Item() item['id'] = hxs.select('//td[@id="item_id"]/text()').re(r'ID: (\d+)') item['name'] = hxs.select('//td[@id="item_name"]/text()').extract() item['description'] = ['id', 'name', 'description'] def parse_row(self, response, row): log.msg('Hi, this is a row!: %r' % row) item = TestItem() item['id'] = row['id'] item['name'] = row['name'] item['description'] = row['description']

Supported options: --callback or -c: spider method to use as callback for parsing the response --rules or -r: use CrawlSpider rules to discover the callback (ie. spider method) to use for parsing the response HtmlXPathSelector(response) item = Item() item['id'] = hxs.select('//td[@id="item_id"]/text()').re(r'ID: (\d+)') item['name'] = hxs.select('//td[@id="item_name"]/text()').extract() 'name', 'description'] def parse_row(self, response, row): log.msg('Hi, this is a row!: %r' % row) item = TestItem() item['id'] = row['id'] item['name'] = row['name']

options: • --callback or -c: spider method to use as callback for parsing the response • --rules or -r: use CrawlSpider rules to discover the callback (ie. spider method) to use for parsing the response HtmlXPathSelector(response) item = Item() item['id'] = hxs.select('//td[@id="item_id"]/text()').re(r'ID: (\d+)') item['name'] = hxs.select('//td[@id="item_name"]/text()').extract() item['description'] = ['id', 'name', 'description'] def parse_row(self, response, row): log.msg('Hi, this is a row!: %r' % row) item = TestItem() item['id'] = row['id'] item['name'] = row['name'] item['description'] = row['description']

options: • --callback or -c: spider method to use as callback for parsing the response • --rules or -r: use CrawlSpider rules to discover the callback (ie. spider method) to use for parsing the response url) sel = Selector(response) item = Item() item[’id’] = sel.xpath(’//td[@id="item_id"]/text()’).re(r’ID: (\d+)’) item[’name’] = sel.xpath(’//td[@id="item_name"]/text()’).extract() item[’description’] = = [’id’, ’name’, ’description’] def parse_row(self, response, row): log.msg(’Hi, this is a row!: %r’ % row) item = TestItem() item[’id’] = row[’id’] item[’name’] = row[’name’] item[’description’] = row[’description’]

options: • --callback or -c: spider method to use as callback for parsing the response • --rules or -r: use CrawlSpider rules to discover the callback (ie. spider method) to use for parsing the response url) sel = Selector(response) item = Item() item[’id’] = sel.xpath(’//td[@id="item_id"]/text()’).re(r’ID: (\d+)’) item[’name’] = sel.xpath(’//td[@id="item_name"]/text()’).extract() item[’description’] = = [’id’, ’name’, ’description’] def parse_row(self, response, row): log.msg(’Hi, this is a row!: %r’ % row) item = TestItem() item[’id’] = row[’id’] item[’name’] = row[’name’] item[’description’] = row[’description’]

Supported options: --callback or -c: spider method to use as callback for parsing the response --rules or -r: use CrawlSpider rules to discover the callback (ie. spider method) to use for parsing the response Selector(response) item = Item() item['id'] = sel.xpath('//td[@id="item_id"]/text()').re(r'ID: (\d+)') item['name'] = sel.xpath('//td[@id="item_name"]/text()').extract() item['description'] 'name', 'description'] def parse_row(self, response, row): log.msg('Hi, this is a row!: %r' % row) item = TestItem() item['id'] = row['id'] item['name'] = row['name']

Supported options: --callback or -c: spider method to use as callback for parsing the response --rules or -r: use CrawlSpider rules to discover the callback (ie. spider method) to use for parsing the response HtmlXPathSelector(response) item = Item() item['id'] = hxs.select('//td[@id="item_id"]/text()').re(r'ID: (\d+)') item['name'] = hxs.select('//td[@id="item_name"]/text()').extract() 'name', 'description'] def parse_row(self, response, row): log.msg('Hi, this is a row!: %r' % row) item = TestItem() item['id'] = row['id'] item['name'] = row['name']

Supported options: --callback or -c: spider method to use as callback for parsing the response --rules or -r: use CrawlSpider rules to discover the callback (ie. spider method) to use for parsing the response Selector(response) item = Item() item['id'] = sel.xpath('//td[@id="item_id"]/text()').re(r'ID: (\d+)') item['name'] = sel.xpath('//td[@id="item_name"]/text()').extract() item['description'] 'name', 'description'] def parse_row(self, response, row): log.msg('Hi, this is a row!: %r' % row) item = TestItem() item['id'] = row['id'] item['name'] = row['name']

through pipelines 24 Chapter 3. Basic concepts Scrapy Documentation, Release 0.24.6 • --rules or -r: use CrawlSpider rules to discover the callback (i.e. spider method) to use for parsing the response %s' % response.url) item = scrapy.Item() item['id'] = response.xpath('//td[@id="item_id"]/text()').re(r'ID: (\d+)') item['name'] = response.xpath('//td[@id="item_name"]/text()').extract() item['description'] = ['id', 'name', 'description'] def parse_row(self, response, row): log.msg('Hi, this is a row!: %r' % row) item = TestItem() item['id'] = row['id'] 36 Chapter 3. Basic concepts Scrapy Documentation

to use as callback for parsing the response --pipelines: process items through pipelines --rules or -r: use CrawlSpider rules to discover the callback (i.e. spider method) to use for parsing the response url) item = scrapy.Item() item['id'] = response.xpath('//td[@id="item_id"]/text()').re(r'ID: (\d+)') item['name'] = response.xpath('//td[@id="item_name"]/text()').extract() 'name', 'description'] def parse_row(self, response, row): log.msg('Hi, this is a row!: %r' % row) item = TestItem() item['id'] = row['id'] item['name'] = row['name']