(or ../ when relevant). scrapy shell index.html will not work as one might expect (and this is by design, not a bug). Because shell favors HTTP URLs over File URIs, and index.html being syntactically similar different values for the same setting, but when they run in the same process it may be impos- sible, by design or because of some limitations, to use these different values. What happens in practice is different crawling speed based on load of both the Scrapy server and the website you are crawling. 5.11.1 Design goals 1. be nicer to sites instead of using default download delay of zero 2. automatically adjust

(or ../ when relevant). scrapy shell index.html will not work as one might expect (and this is by design, not a bug). Because shell favors HTTP URLs over File URIs, and index.html being syntactically similar different values for the same setting, but when they run in the same process it may be impos- sible, by design or because of some limitations, to use these different values. What happens in practice is different crawling speed based on load of both the Scrapy server and the website you are crawling. 5.11.1 Design goals 1. be nicer to sites instead of using default download delay of zero 2. automatically adjust

(or ../ when relevant). scrapy shell index.html will not work as one might expect (and this is by design, not a bug). Because shell favors HTTP URLs over File URIs, and index.html being syntactically similar different values for the same setting, but when they run in the same process it may be impos- sible, by design or because of some limitations, to use these different values. What happens in practice is different crawling speed based on load of both the Scrapy server and the website you are crawling. 5.11.1 Design goals 1. be nicer to sites instead of using default download delay of zero 2. automatically adjust

(or ../ when relevant). scrapy shell index.html will not work as one might expect (and this is by design, not a bug). Because shell favors HTTP URLs over File URIs, and index.html being syntactically similar different values for the same setting, but when they run in the same process it may be impos- sible, by design or because of some limitations, to use these different values. What happens in practice is different crawling speed based on load of both the Scrapy server and the website you are crawling. 5.11.1 Design goals 1. be nicer to sites instead of using default download delay of zero 2. automatically adjust

(or ../ when relevant). scrapy shell index.html will not work as one might expect (and this is by design, not a bug). Because shell favors HTTP URLs over File URIs, and index.html being syntactically similar different values for the same setting, but when they run in the same process it may be impos- sible, by design or because of some limitations, to use these different values. What happens in practice is different crawling speed based on load of both the Scrapy server and the website you are crawling. 5.11.1 Design goals 1. be nicer to sites instead of using default download delay of zero 2. automatically adjust

(or ../ when relevant). scrapy shell index.html will not work as one might expect (and this is by design, not a bug). Because shell favors HTTP URLs over File URIs, and index.html being syntactically similar throttling crawling speed based on load of both the Scrapy server and the website you are crawling. Design goals 1. be nicer to sites instead of using default download delay of zero 2. automatically adjust

(or ../ when relevant). scrapy shell index.html will not work as one might expect (and this is by design, not a bug). Because shell favors HTTP URLs over File URIs, and index.html being syntactically similar the website you are crawling. 5.11. Deploying Spiders 153 Scrapy Documentation, Release 1.1.3 Design goals 1. be nicer to sites instead of using default download delay of zero 2. automatically adjust

(or ../ when relevant). scrapy shell index.html will not work as one might expect (and this is by design, not a bug). Because shell favors HTTP URLs over File URIs, and index.html being syntactically similar throttling crawling speed based on load of both the Scrapy server and the website you are crawling. Design goals 1. be nicer to sites instead of using default download delay of zero 2. automatically adjust

(or ../ when relevant). scrapy shell index.html will not work as one might expect (and this is by design, not a bug). Because shell favors HTTP URLs over File URIs, and index.html being syntactically similar different values for the same setting, but when they run in the same process it may be impos- sible, by design or because of some limitations, to use these different values. What happens in practice is different crawling speed based on load of both the Scrapy server and the website you are crawling. 5.11.1 Design goals 1. be nicer to sites instead of using default download delay of zero 2. automatically adjust

(or ../ when relevant). scrapy shell index.html will not work as one might expect (and this is by design, not a bug). Because shell favors HTTP URLs over File URIs, and index.html being syntactically similar different values for the same setting, but when they run in the same process it may be impos- sible, by design or because of some limitations, to use these different values. What happens in practice is different crawling speed based on load of both the Scrapy server and the website you are crawling. 5.11.1 Design goals 1. be nicer to sites instead of using default download delay of zero 2. automatically adjust