6 More important, if you type response.selector you will access a selector object you can use to query the response, and convenient shortcuts like response.xpath() and response.css() mapping to response occur. xpath(query) Find nodes matching the xpath query and return the result as a SelectorList instance with all ele- ments flattened. List elements implement Selector interface too. query is a string containing the XPATH query to apply. Note: For convenience this method can be called as response.xpath() css(query) Apply the given CSS selector and return a SelectorList instance. query is a string containing

MySpider(Spider): # ... def parse(self, response): sel = Selector(response) # Using XPath query print sel.xpath(’//p’) # Using CSS query print sel.css(’p’) # Nesting queries print sel.xpath(’//div[@foo="bar"]’).css(’span#bold’) 22.0 xpath(query) Find nodes matching the xpath query and return the result as a SelectorList instance with all ele- ments flattened. List elements implement Selector interface too. query is a string string containing the XPATH query to apply. css(query) Apply the given CSS selector and return a SelectorList instance. query is a string containing the CSS selector to apply. In the background, CSS queries

['Quotes to Scrape'] There are two things to note here: one is that we’ve added ::text to the CSS query, to mean we want to select only the text elements directly inside '>] The result of running response ['Quotes to Scrape'] There are two things to note here: one is that we’ve added ::text to the CSS query, to mean we want to select only the text elements directly inside '>] >>> response.xpath("//title/text()").get() 'Quotes

try selecting elements using CSS with the response object: >>> response.css("title") [query='descendant-or-self::title' data=''>] The result of running response ['Quotes to Scrape'] There are two things to note here: one is that we’ve added ::text to the CSS query, to mean we want to select only the text elements directly inside '>] >>> response.xpath("//title/text()").get() 'Quotes

More important, if you type response.selector you will access a selector object you can use to query the response, and convenient shortcuts like response.xpath() and response.css() mapping to response occur. xpath(query) Find nodes matching the xpath query and return the result as a SelectorList instance with all elements flattened. List elements implement Selector interface too. query is a string containing the XPATH query to apply. Note For convenience this method can be called as response.xpath() css(query) Apply the given CSS selector and return a SelectorList instance. query is a string containing

occur. xpath(query) Find nodes matching the xpath query and return the result as a SelectorList instance with all ele- ments flattened. List elements implement Selector interface too. query is a string containing the XPATH query to apply. 48 Chapter 3. Basic concepts Scrapy Documentation, Release 1.0.7 Note: For convenience, this method can be called as response.xpath() css(query) Apply the given CSS CSS selector and return a SelectorList instance. query is a string containing the CSS selector to apply. In the background, CSS queries are translated into XPath queries using cssselect library and run

MySpider(BaseSpider): # ... def parse(self, response): sel = Selector(response) # Using XPath query print sel.xpath(’//p’) # Using CSS query print sel.css(’p’) # Nesting queries print sel.xpath(’//div[@foo="bar"]’).css(’span#bold’) occur. xpath(query) Find nodes matching the xpath query and return the result as a SelectorList instance with all ele- ments flattened. List elements implement Selector interface too. query is a string string containing the XPATH query to apply. css(query) Apply the given CSS selector and return a SelectorList instance. query is a string containing the CSS selector to apply. In the background, CSS queries