. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.7 Item Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 definition (which is included some paragraphs above). 2.1.3 Write a pipeline to store the items extracted Now let’s write an Item Pipeline that serializes and stores the extracted item into a file using pickle: Built-in support for exporting data in multiple formats, including XML, CSV and JSON • A media pipeline for automatically downloading images (or any other media) associated with the scraped items • Support

in an interactive environment. Item Loaders Populate your items with the extracted data. Item Pipeline Post-process and store your scraped data. Built-in services Logging Understand the simple logging class definition (which is included some paragraphs above). Write a pipeline to store the items extracted Now let’s write an Item Pipeline that serializes and stores the extracted item into a file using pickle sources Built-in support for exporting data in multiple formats, including XML, CSV and JSON A media pipeline for automatically downloading images (or any other media) associated with the scraped items Support

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.8 Item Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 for example) or the storage backend (FTP or Amazon S3, for example). You can also write an item pipeline to store the items in a database very easily. 2.1.5 Review scraped data If you check the scraped_data S3, local filesystem) 2.1. Scrapy at a glance 7 Scrapy Documentation, Release 0.12.0 • A media pipeline for automatically downloading images (or any other media) associated with the scraped items • Support

in an interactive environment. Item Loaders Populate your items with the extracted data. Item Pipeline Post-process and store your scraped data. Feed exports Output your scraped data using different backend (FTP or Amazon S3 [http://aws.amazon.com/s3/], for example). You can also write an item pipeline to store the items in a database very easily. Review scraped data If you check the scraped_data formats (JSON, CSV, XML) and storing them in multiple backends (FTP, S3, local filesystem) A media pipeline for automatically downloading images (or any other media) associated with the scraped items Support

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.8 Item Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 for example) or the storage backend (FTP or Amazon S3, for example). You can also write an item pipeline to store the items in a database very easily. 2.1.5 Review scraped data If you check the scraped_data S3, local filesystem) 2.1. Scrapy at a glance 7 Scrapy Documentation, Release 0.18.4 • A media pipeline for automatically downloading images (or any other media) associated with the scraped items • Support

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.8 Item Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 for example) or the storage backend (FTP or Amazon S3, for example). You can also write an item pipeline to store the items in a database very easily. 2.1.5 Review scraped data If you check the scraped_data S3, local filesystem) 2.1. Scrapy at a glance 7 Scrapy Documentation, Release 0.16.5 • A media pipeline for automatically downloading images (or any other media) associated with the scraped items • Support

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 3.8 Item Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 for example) or the storage backend (FTP or Amazon S3, for example). You can also write an item pipeline to store the items in a database very easily. 2.1.5 Review scraped data If you check the scraped_data S3, local filesystem) 2.1. Scrapy at a glance 7 Scrapy Documentation, Release 0.14.4 • A media pipeline for automatically downloading images (or any other media) associated with the scraped items • Support

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.8 Item Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 for example) or the storage backend (FTP or Amazon S3, for example). You can also write an item pipeline to store the items in a database very easily. 2.1.5 Review scraped data If you check the scraped_data S3, local filesystem) 2.1. Scrapy at a glance 7 Scrapy Documentation, Release 0.20.2 • A media pipeline for automatically downloading images (or any other media) associated with the scraped items • Support

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 3.7 Item Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 for example) or the storage backend (FTP or Amazon S3, for example). You can also write an item pipeline to store the items in a database very easily. 2.1.5 Review scraped data If you check the scraped_data formats (JSON, CSV, XML) and storing them in multiple backends (FTP, S3, local filesystem) • A media pipeline for automatically downloading images (or any other media) associated with the scraped items 2.1

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 3.7 Item Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 for example) or the storage backend (FTP or Amazon S3, for example). You can also write an item pipeline to store the items in a database. 6 Chapter 2. First steps Scrapy Documentation, Release 1.3.3 Plus other goodies like reusable spiders to crawl sites from Sitemaps and XML/CSV feeds, a media pipeline for automatically downloading images (or any other media) associated with the scraped items, a caching