Engineering February 2025 6 Introduction When thinking about a large language model input and output, a text prompt (sometimes accompanied by other modalities such as image prompts) is the input the model uses Prompt engineering Remember how an LLM works; it’s a prediction engine. The model takes sequential text as an input and then predicts what the following token should be, based on the data it was trained to do this over and over again, adding the previously predicted token to the end of the sequential text for predicting the following token. The next token prediction is based on the relationship between

Usage + Cost + Loss Growth = Unprecedented • AI Monetization Threats = Rising Competition + Open-Source Momentum + China’s Rise • AI & Physical World Ramps = Fast + Data-Driven • Global Internet User User + Usage + CapEx Growth = Unprecedented Developers in Leading Chipmaker’s Ecosystem 1 2.1 Source: Leading Chipmaker Details on Page 38 AI User + Usage + CapEx Growth = Unprecedented 2.2 Internet mobile app users. App not available in select countries, including China and Russia, as of 5/25. Source: United Nations / International Telecommunications Union (3/25), Sensor Tower (5/25) 0 Years In

maximum generation throughput to 5.76 times. We pretrain DeepSeek-V2 on a high-quality and multi-source corpus consisting of 8.1T tokens, and further perform Supervised Fine-Tuning (SFT) and Reinforcement activated parameters, DeepSeek-V2 and its chat versions still achieve top-tier performance among open-source models. The model checkpoints are available at h t t p s : / / g i t h u b . c o m / d e e p s e (Tokens/Sec) (b) Figure 1 | (a) MMLU accuracy vs. activated parameters, among different open-source models. (b) Training costs and inference efficiency of DeepSeek 67B (Dense) and DeepSeek-V2. Contents

Inference engines DL Compilers Kenrel Libraries Hardware CuDNN NNPack MKL-DNN Hand optimized Open source, automated end-to- end optimization framework for deep learning.TVM Stack High-Level Differentiable (ISA) VTA MicroArchitecture VTA Simulator} compiler, driver, hardware design full stack open source Current TVM Stack VTA Runtime & JIT CompilerTSIM: Support for Future Hardware Current TVM Stack = tvm.IRModule([te_add_one]) print(mod[”te_add_one”].args) Use hybrid script as an alternative text format Directly write pass, manipulate IR structures Accelerate innovation, 
 e.g. use (GA/RL/BayesOpt/your

训练 规划执行多步 骤研究流程 实时调整策略 回溯修正错误 文本 PDF 图像 【多格式数据】 支持搜索多格式数据， 整合多模态信息，生 成带引用和思考过程 总结的报告 Text Text Text “引用” DeepResearch：智能协作，自主研究 表现：人类终极考试，准确率突破 26.6% 这项测试包括3000多个多项选择题和简答题， 涵盖了从语言学到火箭科学、古典文学到生态学的100多个学科。

Q OctoML Open Source at O〇ctoML TVM Meetup 11/8/2019 Jared Roesch OctoML is a new company building DL deployment solutions using the Apache (incubating) TVM project. A goal is to nurture the TVM community and machine learning tr tvm 。 @zxnet 和os 全 W Open Source at OctoML ee We are big believers in the power of open source o 5S$ponsoring multiple employees to contribute to TVML. ee Today

collaborative seamless integration with the ecosystem of AI/ML software frameworks and librariesArm NN open source project ● Linaro-hosted https://www.mlplatform.org/ ● Git and review servers ● Forums and issue

Instruction Buffer Cross Bar Pooling/ EWA© Copyright 2018 Xilinx Xilinx Edge DPU IP (DPUv2) Source: Published results from Huawei 18% 13% 14% 40% 24% 23% 85% 51% 52% 0% 20% 40% 60% 80% 100%

ARM Dot, Nvidia DP4A • Full pipeline is available. Please try it and give suggestions. • Open-source discussions formed the foundations of both the approaches.

(Qualcomm) PART TWO Alios TVM @ ARM CPU AiOS 1驱动万物智能 Alios TVMQOARM CPU 。 Support TFLite ( Open Source and Upstream Master ) 。， Optimize on INT8 & FP32 AiiOS ! 驱动万物智能 Alios TVM @ ARM CPU INT8 * Cache