© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Animesh Jain Amazon SageMaker Neo Compilation of Quantized Models in TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates 𝑟𝑒𝑎𝑙_𝑣𝑎𝑙𝑢𝑒 = 𝑠𝑐𝑎𝑙𝑒 ∗ (𝑞𝑢𝑎𝑛𝑡𝑖𝑧𝑒𝑑_𝑣𝑎𝑙𝑢𝑒 − 𝑧𝑒𝑟𝑜_𝑝𝑜𝑖𝑛𝑡)© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Quantization in TVM • Quantization within pre-quantized graph in TFLite or MxNet • Use high-level wrapper ops of QNN dialect© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. TVM Overview Framework Graph Mxnet TF ….

one can make. The magic of watching AI do your work for you feels like the early days of email and web search – technologies that fundamentally changed our world. The better / faster / cheaper impacts 1993 with release of the World Wide Web (WWW) into the public domain, which allowed users to create websites; however, Tim Berners-Lee invented the World Wide Web in 1989, per CERN. Source: Google, USA Morgan Stanley, ‘Google and Meta: AI vs. Fundamental 2H Debates’ (7/23), Our World in Data, other web sources per MS Years to 50% Adoption of Household Technologies in USA, per Morgan Stanley Consumer

Amazon Web Services, Inc. or its Affiliates. All rights reserved. Presenter: Haichen Shen, Yao Wang Amazon SageMaker Neo, Deep Engine Science Dynamic Model in TVM AWS AI© 2019, Amazon Web Services while loop Limitation of TVM/graph runtime ● Cannot compile and run dynamic models© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Support dynamic model in TVM ● Support Any-dim Graph dispatch for a (sub-)graph In collaboration with Jared Roesch, Zhi Chen, Wei Chen© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Any” in Relay typing Any: represent an unknown

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon/Intel Confidentia Presenter: Zhi Chen, Cody Yu Amazon SageMaker Neo, Deep Engine Science Bring Your Own Codegen to TVM TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Considering You... Design and manufacture a deep learning chip which achieves amazing performance on widely-used operators Suppression (NMS) is too new to be supported by your chip But NMS is supported by TVM!© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Let TVM Be the Compiler of Your Chip Your

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Deploying GluonCV models using TVM© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon with TVM© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Deploy GluonCV Models https://arxiv.org/pdf/1907.02154.pdf© 2019, Amazon Web Services, Inc. or its Affiliates Amazon Trademark Overall Performance AWS DeepLens Acer aiSage NVIDIA Jetson Nano© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Effects of Vision-specific

example of OpenAI’s agentic approach. Leveraging its own virtual browser, Operator can navigate the web, click on buttons, fill in forms, and gather data just like a human would. It can also run processes human intervention, such as: Automating software testing and QA using Operator to interact with web apps 
 like a real user, flagging any UI issues. Updating systems of record on behalf of users, without

can rely on computer-use models to interact directly with those applications and systems through web and application UIs—just as a human would. Each tool should have a standardized definition, enabling the workflow. Query transaction databases or systems like CRMs, read PDF documents, or search the web. Action Enable agents to interact with systems to take actions such as adding new information to

space (~10 lines of Relay IR) - A few days of work - TVM sampling model running in 30us on single server CPU core - Beat hand-written, highly optimized baselines (https://github.com/mozilla/LPCNet) by

the compiled TVM to the SDCard Step 7: Install kernel module cma.ko and run apps/vta_rpc/start_rpc_server.sh Step 8: Configure vta/config/de10nano_config.json to vta_config.json Step 9: Go to vta/hardware/intel

益于其简洁的语法、强大的库支持（如 ）和深度学习框架（如 ）。 此外， ，C++ 有时 用于优化计算密集型任务，而 Java 在企业环境中处理模型部署和系 统集成方面常见。JavaScript 适用于 Web 环境的 LLM 应用。 13 / 32 LLM 基础设施：编程语言 2023 年是大语言模型 (LLM) 之年，Python 作为人工智能领域使用度最高的编程语言，在 2023 年到底有多火？