Infrastructure Improvements to TVM o_uTVM: support for microcontrollers in TVM o_ Virtual Machine and dynamic NNs support (w/ AWS folks) o_ Improved NLP support, with focus on transformers QQ octoML Core loops e Support for different integer division modes, floor division and truncating division. e Unified Object and Node system for TVM runtime o Lays groundwork forimproved multi-language support for expPosing BERT have recently become very Popular and require first class support in TVML. ee What we've done: o Extend the relay ONNX frontend to support all opset versions of BERT. 里This enables importing of native

instructions Use existing documents When creating routines, use existing operating procedures, support scripts, or policy documents to create LLM-friendly routines. In customer service for example, routines both sales and support: Python 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 from agents import Agent, Runner technical_support_agent = Agent( ] ) order_management_agent = Agent( name= , instructions=( "Technical Support Agent", "You provide expert assistance with resolving technical issues, system outages, or product

starting to emerge. AI agents could reshape how users interact with digital systems – from customer support and onboarding to research, scheduling, and internal operations. Enterprises are leading the charge; Act (3/25 = Research Preview Release) Agent Released Select Capabilities • Automated customer support • Case resolution • Lead qualification • Order tracking • Control computer screen directly to estimates and may not align between companies. Oracle Cloud revenue includes Cloud Services & License Support, as well as Cloud License & On-Premise License. IBM Cloud includes all ‘Infrastructure’ line items

(Py/Java/Go) lib = tvm.module.load("mylib.so") func = lib["npufunction0"] func(a, b) Automatic RPC Support remote = tvm.rpc.connect(board_url, port) remote.upload("mylib.so") remote_mod = remote.load_module(“mylib runtime for dynamic models Credit: Jared Roesch, Haichen Shen et.aluTVM: TVM on bare-metal Devices Support bare-metal J-TAG devices, no OS is needed ARM Cortex-M RISC-V Credit: Logan WeberuTVM upcoming: TPUsTensorization Challenge Compute primitives scalar vector tensor Challenge: Build systems to support emerging tensor instructionsTensorization Challenge C = tvm.compute((m, n), 
 lambda y, x:

$40 million in profit improvement, all while maintaining satisfaction scores on par with human support. These results didn’t happen overnight. Klarna achieved this performance by continuously testing easier to customize and fine-tune models—whether as a self-service approach or using our tools and support. We worked closely with Lowe’s, the Fortune 50 home improvement company, to improve the accuracy with AI every day, so we’re often spotting new ways to automate our own work. An example: Our support teams were getting bogged down, spending time accessing systems, trying to understand context, craft

Driver ● Arm Compute Library has been integrated by: ○ MATLAB Coder ○ ONNX RuntimeArm platform support in TVM upstream IPs Target Hardware/Model Options Codegen CPU arm_cpu pixel2 (snapdragon 835), t860) N/A opencl bifrost hikey960 (mali g71) N/A FPGA vta pynq, ultra96 N/A sdaccel Out-of-tree support or WIP: Hexagon DSP (via llvm), Ascend NPU, and more Green: Linaro 96BoardsLinaro for TVM ● Linaro Linaro AI/ML group can be a good fit for TVM collaborations on Arm based platforms to support more devices with various accelerator configurations (from microcontrollers to HPC) by working together with

2019.6 ee 2019.10 Alios TVM Team Set up TFLite Quantized Support 1.61X MobilenetVl TFlite TV Hexagon DSP | Inference Engine DSP (Qualcomm) PART TWO Alios TVM @ ARM CPU AiOS 1驱动万物智能 Alios TVMQOARM CPU 。 Support TFLite ( Open Source and Upstream Master ) 。， Optimize on INT8 & FP32 AiiOS ! 驱动万物智能 Alios TVM could generate HVX instruction 。， Add one Hexagon runtimes named as libtvm_hexagon_runtime.so to support parallel. 。 Could run end-to-end TFLite Mobilenet V2 quantized model on Simulator / Device. /NiiOS

models© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Support dynamic model in TVM ● Support Any-dim in typing ● Use shape function to compute the type at runtime ● Virtual Inc. or its Affiliates. All rights reserved. Dynamic codegen: op dispatch (proposal) ● Goal: support codegen for dynamic shape ● Challenges ○ Single kernel performs poor across different shapes ○

can not only deploy our network, but also get a good performance gain by autotuning 3. TVM can support many kinds of hardware platform: Intel/arm CPU, Nividia/arm GPU, VTA…5 �������������� 1. Get log from step1 on Windows to generate the .dll for deployment 3. For application on 32bits, no support of 32bit tensorflow , a workround from FrozenGene a. python/tvm/contrib/ndk.py options = options

𝒛𝒆𝒓𝒐_𝒑𝒐𝒊𝒏𝒕)© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. How to Support Framework Quantized Operators? Option 1 – Completely add new ops from scratch • New Relay passes rights reserved. Conclusion • TVM community is pursuing both Automatic- and Pre-quantized model support. Contributions are welcomed. • We need new/tuned TVM schedules using fast Integer operations like