Scrapy 1.2 Documentationrestriction – and more • A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler • Plus other goodies like reusable spiders to crawl sites that look like this:“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.” by Albert = quote.css("span.text::text").extract_first() >>> title '“The world as we have created it is a process of our thinking. It cannot be changed ˓→without changing our thinking.”' >>> author = quote.css("small0 码力 | 266 页 | 1.10 MB | 1 年前3 Scrapy 1.3 Documentationrestriction – and more • A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler • Plus other goodies like reusable spiders to crawl sites that look like this:“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.” by Albert = quote.css("span.text::text").extract_first() >>> title '“The world as we have created it is a process of our thinking. It cannot be changed ˓→without changing our thinking.”' >>> author = quote.css("small0 码力 | 272 页 | 1.11 MB | 1 年前3 Scrapy 2.10 Documentationrestriction – and more • A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler • Plus other goodies like reusable spiders to crawl sites that look like this:“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.” by Albert created: >>> text = quote.css("span.text::text").get() >>> text '“The world as we have created it is a process of our thinking. It cannot be changed␣ ˓→without changing our thinking.”' >>> author = quote.css("small0 码力 | 419 页 | 1.73 MB | 1 年前3 Scrapy 2.9 Documentationrestriction – and more • A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler • Plus other goodies like reusable spiders to crawl sites that look like this:“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.” by Albert created: >>> text = quote.css("span.text::text").get() >>> text '“The world as we have created it is a process of our thinking. It cannot be changed␣ ˓→without changing our thinking.”' >>> author = quote.css("small0 码力 | 409 页 | 1.70 MB | 1 年前3 Scrapy 2.8 Documentationrestriction – and more • A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler • Plus other goodies like reusable spiders to crawl sites that look like this:“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.” by Albert created: >>> text = quote.css("span.text::text").get() >>> text '“The world as we have created it is a process of our thinking. It cannot be changed␣ ˓→without changing our thinking.”' >>> author = quote.css("small0 码力 | 405 页 | 1.69 MB | 1 年前3 Scrapy 2.7 Documentationrestriction – and more • A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler • Plus other goodies like reusable spiders to crawl sites that look like this:“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.” by Albert created: >>> text = quote.css("span.text::text").get() >>> text '“The world as we have created it is a process of our thinking. It cannot be changed␣ ˓→without changing our thinking.”' >>> author = quote.css("small0 码力 | 401 页 | 1.67 MB | 1 年前3 Scrapy 0.16 Documentationdatabase very easily. 2.1.5 Review scraped data If you check the scraped_data.json file after the process finishes, you’ll see the scraped items there: [{"url": "http://www.mininova.org/tor/2657665", "name": console running inside your Scrapy process, to introspect and debug your crawler • Logging facility that you can hook on to for catching errors during the scraping process. • Support for crawling based on (see Request class for more info). parse(response) This is the default callback used by Scrapy to process downloaded responses, when their requests don’t specify a callback. The parse method is in charge0 码力 | 203 页 | 931.99 KB | 1 年前3 Scrapy 1.1 Documentationrestriction – and more • A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler • Plus other goodies like reusable spiders to crawl sites that look like this:“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.” by Albert = quote.css("span.text::text").extract_first() >>> title '“The world as we have created it is a process of our thinking. It cannot be changed ˓→without changing our thinking.”' >>> author = quote.css("small0 码力 | 260 页 | 1.12 MB | 1 年前3 Scrapy 1.6 Documentationrestriction – and more • A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler • Plus other goodies like reusable spiders to crawl sites that look like this:“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.” by Albert >>> title = quote.css("span.text::text").get() >>> title '“The world as we have created it is a process of our thinking. It cannot be changed ˓→without changing our thinking.”' >>> author = quote.css("small0 码力 | 295 页 | 1.18 MB | 1 年前3 Scrapy 1.0 Documentationrestriction – and more • A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler • Plus other goodies like reusable spiders to crawl sites repeated) • --callback or -c: spider method to use as callback for parsing the response • --pipelines: process items through pipelines • --rules or -r: use CrawlSpider rules to discover the callback (i.e. spider (see Request class for more info). parse(response) This is the default callback used by Scrapy to process downloaded responses, when their requests don’t specify a callback. The parse method is in charge0 码力 | 244 页 | 1.05 MB | 1 年前3共 62 条- 1
- 2
- 3
- 4
- 5
- 6
- 7