Scrapy 2.10 DocumentationRelease 2.10.1 DOWNLOAD_SLOTS Default: {} Allows to define concurrency/delay parameters on per slot (domain) basis: DOWNLOAD_SLOTS = { "quotes.toscrape.com": {"concurrency": 1, "delay": 2, "randomize_delay":␣ pqueues.DownloaderAwarePriorityQueue does not work together with CONCURRENT_REQUESTS_PER_IP. SCRAPER_SLOT_MAX_ACTIVE_SIZE New in version 2.0. Default: 5_000_000 Soft limit (in bytes) for response data SCHEDULER_DEBUG • SCHEDULER_DISK_QUEUE • SCHEDULER_MEMORY_QUEUE • SCHEDULER_PRIORITY_QUEUE • SCRAPER_SLOT_MAX_ACTIVE_SIZE • SPIDER_CONTRACTS • SPIDER_CONTRACTS_BASE • SPIDER_LOADER_CLASS • SPIDER_LOADER_WARN_ONLY0 码力 | 419 页 | 1.73 MB | 1 年前3
Scrapy 2.9 DocumentationRelease 2.9.0 DOWNLOAD_SLOTS Default: {} Allows to define concurrency/delay parameters on per slot (domain) basis: DOWNLOAD_SLOTS = { "quotes.toscrape.com": {"concurrency": 1, "delay": 2, "randomize_delay":␣ pqueues.DownloaderAwarePriorityQueue does not work together with CONCURRENT_REQUESTS_PER_IP. SCRAPER_SLOT_MAX_ACTIVE_SIZE New in version 2.0. Default: 5_000_000 Soft limit (in bytes) for response data SCHEDULER_DEBUG • SCHEDULER_DISK_QUEUE • SCHEDULER_MEMORY_QUEUE • SCHEDULER_PRIORITY_QUEUE • SCRAPER_SLOT_MAX_ACTIVE_SIZE • SPIDER_CONTRACTS • SPIDER_CONTRACTS_BASE • SPIDER_LOADER_CLASS • SPIDER_LOADER_WARN_ONLY0 码力 | 409 页 | 1.70 MB | 1 年前3
Scrapy 2.11.1 DocumentationRelease 2.11.1 DOWNLOAD_SLOTS Default: {} Allows to define concurrency/delay parameters on per slot (domain) basis: DOWNLOAD_SLOTS = { "quotes.toscrape.com": {"concurrency": 1, "delay": 2, "randomize_delay":␣ pqueues.DownloaderAwarePriorityQueue does not work together with CONCURRENT_REQUESTS_PER_IP. SCRAPER_SLOT_MAX_ACTIVE_SIZE New in version 2.0. Default: 5_000_000 Soft limit (in bytes) for response data SCHEDULER_DEBUG • SCHEDULER_DISK_QUEUE • SCHEDULER_MEMORY_QUEUE • SCHEDULER_PRIORITY_QUEUE • SCRAPER_SLOT_MAX_ACTIVE_SIZE • SPIDER_CONTRACTS • SPIDER_CONTRACTS_BASE • SPIDER_LOADER_CLASS 3.11. Settings0 码力 | 425 页 | 1.76 MB | 1 年前3
Scrapy 2.11 DocumentationRelease 2.11.1 DOWNLOAD_SLOTS Default: {} Allows to define concurrency/delay parameters on per slot (domain) basis: DOWNLOAD_SLOTS = { "quotes.toscrape.com": {"concurrency": 1, "delay": 2, "randomize_delay":␣ pqueues.DownloaderAwarePriorityQueue does not work together with CONCURRENT_REQUESTS_PER_IP. SCRAPER_SLOT_MAX_ACTIVE_SIZE New in version 2.0. Default: 5_000_000 Soft limit (in bytes) for response data SCHEDULER_DEBUG • SCHEDULER_DISK_QUEUE • SCHEDULER_MEMORY_QUEUE • SCHEDULER_PRIORITY_QUEUE • SCRAPER_SLOT_MAX_ACTIVE_SIZE • SPIDER_CONTRACTS • SPIDER_CONTRACTS_BASE • SPIDER_LOADER_CLASS 3.11. Settings0 码力 | 425 页 | 1.76 MB | 1 年前3
Scrapy 2.11.1 DocumentationRelease 2.11.1 DOWNLOAD_SLOTS Default: {} Allows to define concurrency/delay parameters on per slot (domain) basis: DOWNLOAD_SLOTS = { "quotes.toscrape.com": {"concurrency": 1, "delay": 2, "randomize_delay":␣ pqueues.DownloaderAwarePriorityQueue does not work together with CONCURRENT_REQUESTS_PER_IP. SCRAPER_SLOT_MAX_ACTIVE_SIZE New in version 2.0. Default: 5_000_000 Soft limit (in bytes) for response data SCHEDULER_DEBUG • SCHEDULER_DISK_QUEUE • SCHEDULER_MEMORY_QUEUE • SCHEDULER_PRIORITY_QUEUE • SCRAPER_SLOT_MAX_ACTIVE_SIZE • SPIDER_CONTRACTS • SPIDER_CONTRACTS_BASE • SPIDER_LOADER_CLASS • SPIDER_LOADER_WARN_ONLY0 码力 | 425 页 | 1.79 MB | 1 年前3
Scrapy 2.11 Documentationheaders_received signals. DOWNLOAD_SLOTS Default: {} Allows to define concurrency/delay parameters on per slot (domain) basis: DOWNLOAD_SLOTS = { "quotes.toscrape.com": {"concurrency": 1, "delay": 2, "randomize_delay": pqueues.DownloaderAwarePriorityQueue does not work together with CONCURRENT_REQUESTS_PER_IP. SCRAPER_SLOT_MAX_ACTIVE_SIZE New in version 2.0. Default: 5_000_000 Soft limit (in bytes) for response data (scrapy.crawler.Crawler object) engine Crawler.engine attribute spider the active spider slot the engine slot extensions the Extension Manager (Crawler.extensions attribute) stats the Stats Collector0 码力 | 528 页 | 706.01 KB | 1 年前3
Scrapy 2.10 Documentationheaders_received signals. DOWNLOAD_SLOTS Default: {} Allows to define concurrency/delay parameters on per slot (domain) basis: DOWNLOAD_SLOTS = { "quotes.toscrape.com": {"concurrency": 1, "delay": 2, "randomize_delay": pqueues.DownloaderAwarePriorityQueue does not work together with CONCURRENT_REQUESTS_PER_IP. SCRAPER_SLOT_MAX_ACTIVE_SIZE New in version 2.0. Default: 5_000_000 Soft limit (in bytes) for response data (scrapy.crawler.Crawler object) engine Crawler.engine attribute spider the active spider slot the engine slot extensions the Extension Manager (Crawler.extensions attribute) stats the Stats Collector0 码力 | 519 页 | 697.14 KB | 1 年前3
Scrapy 2.9 Documentationheaders_received signals. DOWNLOAD_SLOTS Default: {} Allows to define concurrency/delay parameters on per slot (domain) basis: DOWNLOAD_SLOTS = { "quotes.toscrape.com": {"concurrency": 1, "delay": 2, "randomize_delay": pqueues.DownloaderAwarePriorityQueue does not work together with CONCURRENT_REQUESTS_PER_IP. SCRAPER_SLOT_MAX_ACTIVE_SIZE New in version 2.0. Default: 5_000_000 Soft limit (in bytes) for response data (scrapy.crawler.Crawler object) engine Crawler.engine attribute spider the active spider slot the engine slot extensions the Extension Manager (Crawler.extensions attribute) stats the Stats Collector0 码力 | 503 页 | 686.52 KB | 1 年前3
Scrapy 2.11.1 Documentationheaders_received signals. DOWNLOAD_SLOTS Default: {} Allows to define concurrency/delay parameters on per slot (domain) basis: DOWNLOAD_SLOTS = { "quotes.toscrape.com": {"concurrency": 1, "delay": 2, "randomize_delay": pqueues.DownloaderAwarePriorityQueue does not work together with CONCURRENT_REQUESTS_PER_IP. SCRAPER_SLOT_MAX_ACTIVE_SIZE New in version 2.0. Default: 5_000_000 Soft limit (in bytes) for response data (scrapy.crawler.Crawler object) engine Crawler.engine attribute spider the active spider slot the engine slot extensions the Extension Manager (Crawler.extensions attribute) stats the Stats Collector0 码力 | 528 页 | 706.01 KB | 1 年前3
Scrapy 2.0 Documentationpqueues.DownloaderAwarePriorityQueue does not work together with CONCURRENT_REQUESTS_PER_IP. SCRAPER_SLOT_MAX_ACTIVE_SIZE New in version 2.0. Default: 5_000_000 Soft limit (in bytes) for response data SCHEDULER_DEBUG • SCHEDULER_DISK_QUEUE • SCHEDULER_MEMORY_QUEUE • SCHEDULER_PRIORITY_QUEUE • SCRAPER_SLOT_MAX_ACTIVE_SIZE • SPIDER_CONTRACTS • SPIDER_CONTRACTS_BASE • SPIDER_LOADER_CLASS • SPIDER_LOADER_WARN_ONLY (scrapy.crawler.Crawler object) engine Crawler.engine attribute spider the active spider slot the engine slot extensions the Extension Manager (Crawler.extensions attribute) stats the Stats Collector0 码力 | 336 页 | 1.31 MB | 1 年前3
共 58 条
- 1
- 2
- 3
- 4
- 5
- 6













