Built-in services 143 4.1 Logging 143 4.2 Stats Collection 148 4.3 Sending e-mail 150 4.4 Telnet Console 152 4.5 Web Service 155 5 Solving specific problems 157 5.1 Frequently Asked Questions XPath expressions, with helper methods to extract using regular expressions. • An interactive shell console (IPython aware) for trying out the CSS and XPath expressions to scrape data, very useful when writing user-agent spoofing - robots.txt – crawl depth restriction – and more • A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler •

Sending e-mail Send email notifications when certain events occur. Telnet Console Inspect a running crawler using a built-in Python console. Web Service Monitor and control a crawler using a web service XPath expressions, with helper methods to extract using regular expressions. - An interactive shell console (IPython aware) for trying out the CSS and XPath expressions to scrape data, very useful when writing user-agent spoofing ☐ robots.txt ○ crawl depth restriction ☐ and more - A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler -

Built-in services 147 4.1 Logging 147 4.2 Stats Collection 153 4.3 Sending e-mail 155 4.4 Telnet Console 157 4.5 Web Service 160 5 Solving specific problems 161 5.1 Frequently Asked Questions XPath expressions, with helper methods to extract using regular expressions. • An interactive shell console (IPython aware) for trying out the CSS and XPath expressions to scrape data, very useful when writing user-agent spoofing - robots.txt – crawl depth restriction – and more • A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler •

Built-in services 137 4.1 Logging 137 4.2 Stats Collection 142 4.3 Sending e-mail 144 4.4 Telnet Console 146 4.5 Web Service 149 5 Solving specific problems 151 5.1 Frequently Asked Questions XPath expressions, with helper methods to extract using regular expressions. • An interactive shell console (IPython aware) for trying out the CSS and XPath expressions to scrape data, very useful when writing user-agent spoofing - robots.txt – crawl depth restriction – and more • A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler •

from \[x=11.096259, y=2.479317])|Warn|/sim|15:38:46.3...|/rosout, /t...|/tmp/binar...| 日志输出工具——rqt_console  数据绘图工具——rqt_plot

Built-in services 51 4.1 Logging 51 4.2 Stats Collection 53 4.3 Sending e-mail 57 4.4 Telnet Console 59 4.5 Web Service 61 5 Solving specific problems 69 5.1 Frequently Asked Questions shell console, very useful for writing and debugging your spiders • A built-in Web service for monitoring and controlling your bot • A Telnet console for full unrestricted access to a Python console inside Selectors we’re going to use the built-in Scrapy shell, which also requires IPython (an extended Python console) installed on your system. To start a shell you must go to the project’s top level directory and

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.4 Telnet Console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 monitoring the performance of your spiders and detecting when they get broken • An Interactive shell console for trying XPaths, very useful for writing and debugging your spiders • A System service designed production. • A built-in Web service for monitoring and controlling your bot • A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler • Logging

e-mail Send email notifications when certain events occur. Telnet Console Inspect a running crawler using a built-in Python console. Web Service Monitor and control a crawler using a web service shell console, very useful for writing and debugging your spiders • A built-in Web service for monitoring and controlling your bot - A Telnet console for full unrestricted access to a Python console inside Selectors we’re going to use the built-in Scrapy shell, which also requires IPython (an extended Python console) installed on your system. To start a shell you must go to the project’s top level directory and

Built-in services 125 4.1 Logging 125 4.2 Stats Collection 129 4.3 Sending e-mail 131 4.4 Telnet Console 133 4.5 Web Service 136 5 Solving specific problems 137 5.1 Frequently Asked Questions XPath expressions, with helper methods to extract using regular expressions. • An interactive shell console (IPython aware) for trying out the CSS and XPath expressions to scrape data, very useful when writing user-agent spoofing - robots.txt – crawl depth restriction – and more • A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler •

Built-in services 61 4.1 Logging 61 4.2 Stats Collection 63 4.3 Sending e-mail 67 4.4 Telnet Console 69 4.5 Web Service 71 5 Solving specific problems 79 5.1 Frequently Asked Questions monitoring the performance of your spiders and detecting when they get broken • An Interactive shell console for trying XPaths, very useful for writing and debugging your spiders • A System service designed production. • A built-in Web service for monitoring and controlling your bot • A Telnet console for hooking into a Python console running inside your Scrapy process, to introspect and debug your crawler •