：这些框架经过优化，以充分利用 GPU、TPU 等高性能计算硬件，以加速模型 的训练和推理过程。 ：为了处理大型数据集和大规模参 数网络，这些框架通常设计得易于水平扩展， 支持在多个处理器或多个服务器上并行处理。 ：它们提供工具来有效地加 载、处理和迭代大型数据集，这对于训练大 型模型尤为重要。 国产深度学习框架 OneFlow 架构 （图源：https://www.oneflow.or 益于其简洁的语法、强大的库支持（如 ）和深度学习框架（如 ）。 此外， ，C++ 有时 用于优化计算密集型任务，而 Java 在企业环境中处理模型部署和系 统集成方面常见。JavaScript 适用于 Web 环境的 LLM 应用。 13 / 32 LLM 基础设施：编程语言 2023 年是大语言模型 (LLM) 之年，Python 作为人工智能领域使用度最高的编程语言，在 2023 年到底有多火？

DeepSeek-R1-Q4_K_M 404 GB ≥500 GB ⾼性能服务器/云GPU 下载地址： HuggingFace模型库 Unsloth AI官⽅说明 2. 硬件配置建议 硬件类型 推荐配置 性能表现（短⽂本⽣成） 消费级设备 Mac Studio（192GB统⼀内存） 10+ token/秒 ⾼性能服务器 4×RTX 4090（96GB显存+384GB内存） 7-8 token/秒（混合推理）

• 安全人才规模全球领先 • 漏洞挖掘能力全球领先 四个全球领先 世界的360 • 实战能力第一，实战是检验安全企业能力的唯一标准 • 安全研发投入第一，相当于第2名到第10名的总和 • 服务器和算力投入安全行业第一 • 创新能力第一，专利申请1.5万件，安全行业最多 • 服务和响应能力第一 • 用户数量第一，覆盖225个国家和地区的15亿终端 • 企业客户规模第一的网安公司 • 件分析平台 分析研判平台 端 • 服务全球15亿终端 • 覆盖全球225个国家 和地区 终端探针密布 云 数 智 知识 人 云端数据汇聚 • 探针数据上传到云 端 • 20万台服务器 • 210个数据中心 • 4000G出口带宽 • 1000P算力规模 大数据分析 • 总规模2.2EB，每天新 增1.5PB • 测绘数据300亿条 • 90亿+域名信息 • 存活网址库总量50000

口协议、性能评定、试验方法等技术要求，包括智能传感器的架 构、指令、数据格式、信息提取方法、信息融合方法、功能集成 方法、性能指标和评价方法等标准。 4. 计算设备标准。规范人工智能加速卡、人工智能加速模 组、人工智能服务器等计算设备，及使能软件的技术要求和测试 方法，包括人工智能计算设备虚拟化方法，人工智能加速模组接 口协议和测试方法，及使能软件的访问协议、功能、性能、能效 的测试方法和运行维护要求等标准。 5

Amazon Web Services, Inc. or its Affiliates. All rights reserved. Presenter: Haichen Shen, Yao Wang Amazon SageMaker Neo, Deep Engine Science Dynamic Model in TVM AWS AI© 2019, Amazon Web Services while loop Limitation of TVM/graph runtime ● Cannot compile and run dynamic models© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Support dynamic model in TVM ● Support Any-dim Graph dispatch for a (sub-)graph In collaboration with Jared Roesch, Zhi Chen, Wei Chen© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Any” in Relay typing Any: represent an unknown

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon/Intel Confidentia Presenter: Zhi Chen, Cody Yu Amazon SageMaker Neo, Deep Engine Science Bring Your Own Codegen to TVM TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Considering You... Design and manufacture a deep learning chip which achieves amazing performance on widely-used operators Suppression (NMS) is too new to be supported by your chip But NMS is supported by TVM!© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Let TVM Be the Compiler of Your Chip Your

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Animesh Jain Amazon SageMaker Neo Compilation of Quantized Models in TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates 𝑟𝑒𝑎𝑙_𝑣𝑎𝑙𝑢𝑒 = 𝑠𝑐𝑎𝑙𝑒 ∗ (𝑞𝑢𝑎𝑛𝑡𝑖𝑧𝑒𝑑_𝑣𝑎𝑙𝑢𝑒 − 𝑧𝑒𝑟𝑜_𝑝𝑜𝑖𝑛𝑡)© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Quantization in TVM • Quantization within pre-quantized graph in TFLite or MxNet • Use high-level wrapper ops of QNN dialect© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. TVM Overview Framework Graph Mxnet TF ….

one can make. The magic of watching AI do your work for you feels like the early days of email and web search – technologies that fundamentally changed our world. The better / faster / cheaper impacts 1993 with release of the World Wide Web (WWW) into the public domain, which allowed users to create websites; however, Tim Berners-Lee invented the World Wide Web in 1989, per CERN. Source: Google, USA Morgan Stanley, ‘Google and Meta: AI vs. Fundamental 2H Debates’ (7/23), Our World in Data, other web sources per MS Years to 50% Adoption of Household Technologies in USA, per Morgan Stanley Consumer

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Deploying GluonCV models using TVM© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon with TVM© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Deploy GluonCV Models https://arxiv.org/pdf/1907.02154.pdf© 2019, Amazon Web Services, Inc. or its Affiliates Amazon Trademark Overall Performance AWS DeepLens Acer aiSage NVIDIA Jetson Nano© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Effects of Vision-specific

example of OpenAI’s agentic approach. Leveraging its own virtual browser, Operator can navigate the web, click on buttons, fill in forms, and gather data just like a human would. It can also run processes human intervention, such as: Automating software testing and QA using Operator to interact with web apps 
 like a real user, flagging any UI issues. Updating systems of record on behalf of users, without