Apache RocketMQ – Trillion Messaging in Practice 周新宇(花名：尘央) © 2 0 1 7 A l i b a b a M i d d l e w a r e G r o u p PROFILE MCS, has rich experience in distributed system design and performance

includes smartphones, IOT devices, robotics, etc. Source: Weiss et al. ‘AI Index: Mapping the $4 Trillion Enterprise Impact’ via Morgan Stanley (10/23) Enabling Infrastructure CPUs Big Data / Cloud in 5/25, ‘This time last year, we were processing 9.7 trillion tokens a month across our products and APIs. Now, we’re processing over 480 trillion — that’s 50 times more…Over 7 million developers are Next Door This time last year, we were processing 9.7 trillion tokens a month across our products and APIs. Now, we’re processing over 480 trillion – that’s 50 times more. - Google I/O 2025 Press Release

一般用于描述模型的参数数量。例如，具有 50B 参数的模型代表这个 模型有 50 亿个参数。Ollama3 有尺寸 8B 和 70B，Phi-3-mini 有 3.8B 参数等。 T（万亿，Trillion）：在英文里是 Trillion 的缩写，表示万亿。在 AI 大模型 中，”T” 常用来表示模型在训练中处理的 Token 数量。Token 是指模型处理的 基本单元，可以是一个单词、子词，或者字符等。 在大

FLOPs Utilization (MFU). During our practical training on the H800 cluster, for training on each trillion tokens, DeepSeek 67B requires 300.6K GPU hours, while DeepSeek-V2 needs only 172.8K GPU hours, i preprint arXiv:2404.04475, 2024. W. Fedus, B. Zoph, and N. Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. CoRR, abs/2101.03961, 2021. URL https://arxiv

Software Fail Watch documents 606 software failures in CY 2017 • 3.6 billion people affected • $1.7 trillion lost revenue • Software failures resulted in 268 years of downtime • The number of reported failures

1 μs 3 μs 10 μs 34 μs 100 μs 1 ms 1 ms 3.6 seconds 100 1 μs 6 μs 100 μs 665 μs 10 ms 1 sec >400 trillion centuries >googol centuries 1,000 1 μs 9 μs 1 ms ~10 ms 1 sec 16.67 min ... ... 10,000 1 μs

Datasets 能力的系統 Apache Drill 是 Dremel 的 Open Source 版本，號稱可以 Scale Out 到 10,000+ 的 Node，處 理 PB 等級的資料，秒殺 Trillion 數量級的 Record Google BigQuery 就是提供 Dremel 的 IaaS Dremel、Drill、BigQuery What is BigQuery, Its Features

multi-domain ctr prediction. CIKM 2021. [12] Fedus W, Zoph B, Shazeer N. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. arXiv:2101.03961, 2021. [13] Zoph B, Bello