platform, introduced a new AI assistant to streamline customer service. Within a few months, the assistant was handling two-thirds of all service chats—doing the work of hundreds of agents and cutting average invested heavily in our API to make it easier to customize and fine-tune models—whether as a self-service approach or using our tools and support. We worked closely with Lowe’s, the Fortune 50 home improvement team Uses it to answer 40,000 questions a year on policies, compliance, and more. The Customer Service team Automates the sentiment analysis of NPS surveys. 16 AI in the EnterpriseAnd the wins continue

Although training an MoE model will introduce additional commu- nication overheads, through our operator and communication optimizations, the training for DeepSeek-V2 can attain a relatively high Model compared with dense DeepSeek 67B. Inference Efficiency. In order to efficiently deploy DeepSeek-V2 for service, we first convert its parameters into the precision of FP8. In addition, we also perform KV cache DeepSeek-V2 based on the prompt and generation length distribution from the actually deployed DeepSeek 67B service. On a single node with 8 H800 GPUs, DeepSeek-V2 achieves a generation throughput exceeding 50K tokens

Admin Costs Margins Marketing Spend Effectivity ROIC Revenues Sales Productivity Customer Service Production / Output Revenue-Focused Cost-Focused ‘Traditional’ Enterprise AI Adoption = Rising Evolution = Chat Responses → Doing Work A new class of AI is now emerging – less assistant, more service provider. What began as basic conversational interfaces may now be evolving into something far (1/25), Amazon (3/25) AI Agent Deployments = AI Incumbent Product Launches Accelerating OpenAI Operator (1/25 = Research Preview Release) Salesforce Agentforce (10/24 = General Release) Anthropic Claude

subgraphs 1. Implement an operator-level annotator, OR 2. Implement a graph-level annotator© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Option 1: Operator-Level Annotation ● Implement Boolean functions in the template def conv2d(attrs, args): return is_float32(args) Relay operator name Operator attributes and args (inputs) can be checked as well Return True/False for this op After Device General Devices (CPU/GPU/FPGA) Mark supported operators or subgraphs 1. Implement extern operator functions, OR 2. Implement a graph annotator© 2019, Amazon Web Services, Inc. or its Affiliates

scratch • New Relay passes and TVM schedules required • AlterOpLayout, Graph Fusion etc require work/operator • No reuse of existing Relay and TVM infrastructure. Option 2 – Lower to a sequence of existing 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lowering of QNN Quantize Operator fn (%input_data: Tensor[(2, 5), float32]) { qnn.quantize(%input_data, out_dtype="uint8", output_zero_point=127 Affiliates. All rights reserved. QNN Conv2D Operator • Calculations are different from FP32 Conv2D https://discuss.tvm.ai/t/tf-lite-quantized-conv2d-operator-conversion/2651/8 𝑟𝑒𝑎𝑙_𝑣𝑎𝑙𝑢𝑒 = 𝒔𝒄𝒂𝒍𝒆

function ● Relax type inference/checking for Any at compilation time ● Register a shape function for operator to check the type and compute the output shape© 2019, Amazon Web Services, Inc. or its Affiliates function ● Relax type inference/checking for Any at compilation time ● Register a shape function for operator to check the type and compute the output shape ● Shape function has two modes (op_attrs, input_tensors function ● Relax type inference/checking for Any at compilation time ● Register a shape function for operator to check the type and compute the output shape ● Shape function has two modes (op_attrs, input_tensors

cuDNN Offload to heavily optimized DNN operator library FrameworksLimitations of Existing Approach cuDNN Frameworks New operator introduced by operator fusion optimization potential benefit: System High-level data flow graph and optimizations Directly generate optimized program for new operator workloads and hardware Hardware FrameworksWhy Automation is the Future Clear winner on

3400us (baseline), 40us (target) - 85x speedup - Uh ohEnter, TVM and model co-design - PyTorch operator overhead makes interpreter infeasible - Reduce FLOPs with block-sparsified weight matrices -

`f` string syntax for string interpolation is more readable and concise than the traditional `+` operator. 4. The code doesn’t handle errors that might occur during the renaming process. It would be better

sequence of steps that must be executed to meet the user’s goal, whether that's resolving a customer service issue, booking a restaurant reservation, committing a code change, 
 or generating a report. Applications judgment, exceptions, or 
 context-sensitive decisions, for example refund approval 
 in customer service workflows. 02 Difficult-to-maintain rules: Systems that have become unwieldy due to extensive and records, or sending messages. Send emails and texts, update a CRM record, hand-off a customer service ticket to a human. Orchestration Agents themselves can serve as tools for other agents—see the