layers of AI infrastructure: agentic interfaces, enterprise copilots, real-world autonomous systems, and sovereign models. Rapid advances in artificial intelligence, compute infrastructure, and global ‘AI Index: Mapping the $4 Trillion Enterprise Impact’ via Morgan Stanley (10/23) Enabling Infrastructure CPUs Big Data / Cloud GPUs Computing Cycles Over Time – 1960s-2020s, per Morgan Stanley we access and move information is happening much faster… …AI is a compounder – on internet infrastructure, which allows for wicked-fast adoption of easy-to-use broad-interest services. AI Technology

Apache (incubating) TVM project. A goal is to nurture the TVM community and contribute new infrastructure and features. octom|.ai @octoml Q octoML Founding Team - The Octonauts - 四人全外日 Luis Ceze employees to contribute to TVML. ee Today we'ltouch on a few of those contribution areas: o Core Infrastructure Improvements to TVM o_uTVM: support for microcontrollers in TVM o_ Virtual Machine and dynamic o_ Improved NLP support, with focus on transformers QQ octoML Core Infrastructure Refactors ee New Integer Analysis Infrastructure o_ Supports the ability to handle nested division and modulus o_ Improves

LLVM, Cuda, C, … Framework Parsers Graph level optimizations Tensor-level optimizations Machine code generation© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Quantization and TVM infrastructure. Option 2 – Lower to a sequence of existing Relay operators • We introduced a new Relay dialect – QNN to encapsulate this work • Complete reuse of Relay pass infrastructure • Possible

often prune models to 80%+ sparsity(with retraining) - Massive speedups combined with specialized code-generation techniques (TVM, Xbyak, etc) - Interesting new tradeoffs - how const are parameters FBGEMMPyTorch and TVM - Lots of opportunity in PyTorch - Graph optimization - Existing fusion infrastructure fairly limited (CUDA-only, injective-only) - Kernel synthesis - Dynamic shapes, stride specialization

faster—a win for everyone. We see a lot of opportunity to continue to invest 
 in this new infrastructure in ways that will help 
 us grow revenue. Chris Hyams CEO 10 AI in the EnterpriseLesson 3 Developers now build consistently 
 high-quality apps, faster, without having to get into the source code. Security, guardrails, and routing logic are all built in. 18 AI in the EnterpriseAs a result,

2023 年前四个月，向量数据库公司融资额 ，超过了 2022 年的总和 （图源：https://www.cbinsights.com/research/generative-ai-infrastructure- vector-database/） 7 / 32 LLM 基础设施：大模型框架及微调 (Fine Tuning) 大模型框架指专门设计用于构建、训练和部署大型机器 学习模型和深度学习模型的软件框架。这些框架提供了 22 / 32 AI 编程新形态：代码生成工具 通过原型或图片直接生成包含代码的完整页面， 。 、 、 都是该形态出色的产品。 tldraw v0.dev Screenshot to code 23 / 32 LLM Agent（AI Agent） LLM Agent 是一种基于 LLM 的智能代理，它能够自主学习和执行任务， 具有一定的“认知能力和决策能力”。LLM Agent

Prompt Engineering 40 Code prompting 42 Prompts for writing code 42 Prompts for explaining code 44 Prompts for translating code 46 Prompts for debugging and reviewing code 48 What about multimodal information extraction, question and answering, text classification, language or code translation, code generation, and code documentation or reasoning. Please feel free to refer to Google’s prompting guides2 ‘providing an additional task to the system’. For example, you could use a system prompt to generate a code snippet that is compatible with a specific programming language, or you could use a system prompt

. . . 31 E Discussion About Pre-Training Data Debiasing 32 F Additional Evaluations on Math and Code 33 G Evaluation Formats 34 3 1. Introduction In the past few years, Large Language Models (LLMs) corpus. Then, we collect 1.5M conversational sessions, which encompass various domains such as math, code, writing, reasoning, safety, and more, to perform Supervised Fine-Tuning (SFT) for DeepSeek-V2 Chat include GSM8K (Cobbe et al., 2021), MATH (Hendrycks et al., 2021), and CMath (Wei et al., 2023). Code datasets include HumanEval (Chen et al., 2021), MBPP (Austin et al., 2021), and CRUXEval (Gu et al

whether that's resolving a customer service issue, booking a restaurant reservation, committing a code change, 
 or generating a report. Applications that integrate LLMs but don’t use them to control Explicit guidelines and guardrails defining how the 
 agent behaves Here’s what this looks like in code when using OpenAI’s Agents SDK. You can also implement the same concepts using your preferred library learning of specialized domain-specific languages. In contrast, the Agents SDK adopts a more flexible, code-first approach. Developers can 
 directly express workflow logic using familiar programming constructs

统，以便你可以使用它创建和运行 Docker 容器。 然后再运行一条命令就可以了： docker run -d --name lobe-chat -p 10084:3210 -e ACCESS_CODE=lobe66 lobehub/lobe-chat:latest 22 解释下这条命令，它用于以守护进程模式（后台）运行一个名为 lobe-chat 的 Docker 容器，并设置一些特定参数： 容 器 的 3210 端 口。 这 样， 主 机 的 10084 端 口 的 请 求 会 被 转 发 到 容 器 的 3210 端 口。 -e ACCESS_CODE=lobe66 ： 设 置 环 境 变 量 ACCESS_CODE 的 值 为 lobe66 ， 这 通 常 是 用 于 在 容 器 内 配 置 应 用 程 序 的 参 数。 lobehub/lobe-chat:latest ：