208 6.3 Spider Middleware ..... 224 6.4 Extensions ..... 231 6.5 Core API ..... 237 6.6 Signals ..... 245 6.7 Item Exporters ..... 251 7 All the rest ..... 259 7.1 Release notes ..... 259 declarations. • Strong extensibility support, allowing you to plug in your own functionality using signals and a well-defined API (middlewares, extensions, and pipelines). • Wide range of built-in extensions lot of components in the project for their single entry access (such as extensions, middleware, signals managers, etc). See Crawler API to know more about them. ## settings Configuration for running

INFO Q Cluster-31781 running. Stopping the cluster with ctrl-c or either the SIGTERM and SIGKILL signals, will initiate the Stop procedure: 16:44:12 [Q] INFO Q Cluster-31781 stopping. 16:44:12 [Q] INFO can quickly move on to serving web pages to your user. #### 1.13.2 Signals A good place to use async tasks are Django’s model signals. You don’t want to delay the saving or creation of objects, but sometimes actions: # Message on object change from Django.contrib.auth.models import User from Django.db.models.signals import pre_save from Django.dispatch import receiver from Django_q.tasks import async # set up the

7 Reference ..... 143 7.1 Requests and Responses ..... 143 7.2 Settings ..... 150 7.3 Signals ..... 161 7.4 Exceptions ..... 164 7.5 Item Exporters ..... 166 8 All the rest ..... 173 with the scraped items • Support for extending Scrapy by plugging your own functionality using signals and a well-defined API (middlewares, extensions, and pipelines). • Wide range of built-in middleware items have an unique id, but our spider returns multiple items with the same id: from scrapy import signals from scrapy.exceptions import DropItem class DuplicatesPipeline(object): def __init__(self):

responses. Settings Learn how to configure Scrapy and see all available settings. Signals See all available signals and how to work with them. Exceptions See all available exceptions and their meaning with the scraped items - Support for extending Scrapy.by plugging your own functionality using signals and a well-defined API (middlewares, extensions, and pipelines). • Wide range of built-in middlewares items have an unique id, but our spider returns multiple items with the same id: from scrapy import signals from scrapy.exceptions import DropItem class DuplicatesPipeline(object): def __init__(self):

7 Reference ..... 139 7.1 Requests and Responses ..... 139 7.2 Settings ..... 146 7.3 Signals ..... 157 7.4 Exceptions ..... 160 7.5 Item Exporters ..... 161 8 All the rest ..... 169 with the scraped items • Support for extending Scrapy by plugging your own functionality using signals and a well-defined API (middlewares, extensions, and pipelines). • Wide range of built-in middleware items have an unique id, but our spider returns multiple items with the same id: from scrapy import signals from scrapy.exceptions import DropItem class DuplicatesPipeline(object): def __init__(self):

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 7.3 Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Documentation, Release 0.24.6 • Support for extending Scrapy by plugging your own functionality using signals and a well-defined API (mid- dlewares, extensions, and pipelines). • Wide range of built-in middlewares information see Logging. closed(reason) Called when the spider closes. This method provides a shortcut to signals.connect() for the spider_closed signal. Spider example Let’s see an example: import scrapy class

responses. Settings Learn how to configure Scrapy and see all available settings. Signals See all available signals and how to work with them. Exceptions See all available exceptions and their meaning with the scraped items - Support for extending Scrapy.by plugging your own functionality using signals and a well-defined API (middlewares, extensions, and pipelines). • Wide range of built-in middlewares items have an unique id, but our spider returns multiple items with the same id: from scrapy import signals from scrapy.exceptions import DropItem class DuplicatesPipeline(object): def __init__(self):

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 6.6 Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . larations. • Strong extensibility support, allowing you to plug in your own functionality using signals and a well-defined API (middlewares, extensions, and pipelines). • Wide range of built-in extensions lot of components in the project for their single entry access (such as extensions, middlewares, signals managers, etc). See Crawler API to know more about them. settings Configuration for running this

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 6.6 Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . larations. • Strong extensibility support, allowing you to plug in your own functionality using signals and a well-defined API (middlewares, extensions, and pipelines). • Wide range of built-in extensions lot of components in the project for their single entry access (such as extensions, middlewares, signals managers, etc). See Crawler API to know more about them. settings Configuration for running this

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 6.6 Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . larations. • Strong extensibility support, allowing you to plug in your own functionality using signals and a well-defined API (middlewares, extensions, and pipelines). • Wide range of built-in extensions lot of components in the project for their single entry access (such as extensions, middlewares, signals managers, etc). See Crawler API to know more about them. settings Configuration for running this