Selectors 46 3.4 Items 64 3.5 Item Loaders 70 3.6 Scrapy shell 78 3.7 Item Pipeline 83 3.8 Feed exports 87 3.9 Requests and Responses 95 3.10 Link Extractors 109 3.11 Settings 111 3.12 Exceptions ... 224 6.4 Extensions ..... 231 6.5 Core API ..... 237 6.6 Signals ..... 245 6.7 Item Exporters ..... 251 7 All the rest ..... 259 7.1 Release notes ..... 259 7.2 Contributing to Scrapy using an auto-throttling extension that tries to figure out these automatically. Note: This is using feed exports to generate the JSON file, you can easily change the export format (XML or CSV, for example)

## 从高并发到极端并发： 百度Feed与春晚红包的高可用实践 吴永巍 百度 主任架构师 # TGO鲲鹏会 # 汇聚全球科技领导者的高端社群 全球12大城市 850+高端科技领导者 使命 Mission 为社会输送更多优秀的 科技领导者 ## 愿景 Vision 构建全球领先的有技术背景 优秀人才的学习成长平台  ## 自我介绍 • 06年加入百度，主任架构师 • Feed主任架构师，百度春晚项目总架构师 - 重点关注领域： ✓推荐系统 ✓搜索引擎 ✓分布式存储与计算、大数据与机器学习 √大规模系统设计与优化…  ## 目录 • 春晚项目，技术挑战 - 整体拆解，架构设计 - 各子系统高可用设计 • Feed信息流：常规到极端 全方位工程实践 ## 春晚，极端并发，技术实力最高级别的检验 • 春晚的力量 • 业界技术难题 现场直播，没有重来的机会 - 不仅仅是摇一摇红包 ✓信息流 + 视频 ✓语音 + 搜索

– Accès unique à plusieurs sources d'informations - Mises à jour fréquentes ## Historique • Feed (fil ou flux) RSS • Format d'échange de données en XML – fournir ou recueillir des données structurées 2008 09:39:21 GMT ### Atom 1.0 <feed xmlns="http://www.w3.org/2005/Atom"> version Atom ch/atom/1234 2008-10-27T18:30:02Z

Un tout nouveau cours.

feed> <feed> ## Générer le flux • Données dynamiques • Source de données identique à celle de l'application

Selectors 46 3.4 Items 65 3.5 Item Loaders 69 3.6 Scrapy shell 80 3.7 Item Pipeline 85 3.8 Feed exports 89 3.9 Requests and Responses 95 3.10 Link Extractors 108 3.11 Settings 110 3.12 Exceptions ... 219 6.4 Extensions ..... 225 6.5 Core API ..... 231 6.6 Signals ..... 240 6.7 Item Exporters ..... 245 7 All the rest ..... 253 7.1 Release notes ..... 253 7.2 Contributing to Scrapy using an auto-throttling extension that tries to figure out these automatically. Note: This is using feed exports to generate the JSON file, you can easily change the export format (XML or CSV, for example)

Selectors 44 3.4 Items 61 3.5 Item Loaders 67 3.6 Scrapy shell 76 3.7 Item Pipeline 80 3.8 Feed exports 84 3.9 Requests and Responses 94 3.10 Link Extractors 108 3.11 Settings 111 3.12 Exceptions ... 223 6.4 Extensions ..... 230 6.5 Core API ..... 235 6.6 Signals ..... 244 6.7 Item Exporters ..... 250 7 All the rest ..... 257 7.1 Release notes ..... 257 7.2 Contributing to Scrapy using an auto-throttling extension that tries to figure out these automatically. Note: This is using feed exports to generate the JSON file, you can easily change the export format (XML or CSV, for example)

Selectors 48 3.4 Items 66 3.5 Item Loaders 72 3.6 Scrapy shell 81 3.7 Item Pipeline 85 3.8 Feed exports 90 3.9 Requests and Responses 102 3.10 Link Extractors 122 3.11 Settings 124 3.12 6.3 Spider Middleware 247 6.4 Extensions 254 6.5 Signals 260 6.6 Scheduler 267 6.7 Item Exporters 269 6.8 Components 276 6.9 Core API 278 7 All the rest 289 7.1 Release notes 289 7 using an auto-throttling extension that tries to figure out these automatically. Note: This is using feed exports to generate the JSON file, you can easily change the export format (XML or CSV, for example)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 3.8 Feed exports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 6.7 Item Exporters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 using an auto-throttling extension that tries to figure out these automatically. Note: This is using feed exports to generate the JSON file, you can easily change the export format (XML or CSV, for example)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 3.8 Feed exports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 6.7 Item Exporters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 using an auto-throttling extension that tries to figure out these automatically. Note: This is using feed exports to generate the JSON file, you can easily change the export format (XML or CSV, for example)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 3.8 Feed exports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 6.7 Item Exporters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 using an auto-throttling extension that tries to figure out these automatically. Note: This is using feed exports to generate the JSON file, you can easily change the export format (XML or CSV, for example)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 3.8 Feed exports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 6.7 Item Exporters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 using an auto-throttling extension that tries to figure out these automatically. Note: This is using feed exports to generate the JSON file, you can easily change the export format (XML or CSV, for example)