Amazon Web Services, Inc. or its Affiliates. All rights reserved. Presenter: Haichen Shen, Yao Wang Amazon SageMaker Neo, Deep Engine Science Dynamic Model in TVM AWS AI© 2019, Amazon Web Services while loop Limitation of TVM/graph runtime ● Cannot compile and run dynamic models© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Support dynamic model in TVM ● Support Any-dim Graph dispatch for a (sub-)graph In collaboration with Jared Roesch, Zhi Chen, Wei Chen© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Any” in Relay typing Any: represent an unknown

one can make. The magic of watching AI do your work for you feels like the early days of email and web search – technologies that fundamentally changed our world. The better / faster / cheaper impacts 1993 with release of the World Wide Web (WWW) into the public domain, which allowed users to create websites; however, Tim Berners-Lee invented the World Wide Web in 1989, per CERN. Source: Google, USA Morgan Stanley, ‘Google and Meta: AI vs. Fundamental 2H Debates’ (7/23), Our World in Data, other web sources per MS Years to 50% Adoption of Household Technologies in USA, per Morgan Stanley Consumer

example of OpenAI’s agentic approach. Leveraging its own virtual browser, Operator can navigate the web, click on buttons, fill in forms, and gather data just like a human would. It can also run processes human intervention, such as: Automating software testing and QA using Operator to interact with web apps 
 like a real user, flagging any UI issues. Updating systems of record on behalf of users, without ensuring internal governance and compliance. Flexible retention Adjust settings for logging and storage to match your organization’s policies. For more on OpenAI and security, visit our Security page

Over the past several decades, with the explosive growth of renewable energy, large-scale energy storage technologies allow intermittent renewable energy to replace traditional energy. High-performance promising candidates for large-scale energy storage intermittent technologies. Since commercialization, lithium-ion batteries (LIBs)have become mainstream energy storage devices with their high output voltage electronic conduction network within the electrode,ultimately resulting in a sharp decline in Li+ storage capacity and attenuation of cycle life. ln order to overcome these problems, previous research

Tet tl 引 -。 Let t2 3 memory planning,， storage Let s = alLLoc_storage(40，64，f32) ; Tet outl = attoc_tensor(s，(19,)，f32); coalescing

MLA, respectively. The amount of KV cache is measured by the number of elements, regardless of the storage precision. For DeepSeek-V2, ?? is set to 4?ℎ and ?? ℎ is set to ?ℎ 2 . So, its KV cache is equal parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–16. IEEE, 2020. C. Riquelme, J. Puigcerver, B. Mustafa, M. Neumann, R. Jenatton

Vectorized load/store for higher bandwidth utilization 。Double buffer to hide memory load latency 。 storage align to reduce bank conflicts of shared memory 。 Virtual threads for data reuse (on going) Performance

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon/Intel Confidentia Presenter: Zhi Chen, Cody Yu Amazon SageMaker Neo, Deep Engine Science Bring Your Own Codegen to TVM TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Considering You... Design and manufacture a deep learning chip which achieves amazing performance on widely-used operators Suppression (NMS) is too new to be supported by your chip But NMS is supported by TVM!© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Let TVM Be the Compiler of Your Chip Your

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Animesh Jain Amazon SageMaker Neo Compilation of Quantized Models in TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates 𝑟𝑒𝑎𝑙_𝑣𝑎𝑙𝑢𝑒 = 𝑠𝑐𝑎𝑙𝑒 ∗ (𝑞𝑢𝑎𝑛𝑡𝑖𝑧𝑒𝑑_𝑣𝑎𝑙𝑢𝑒 − 𝑧𝑒𝑟𝑜_𝑝𝑜𝑖𝑛𝑡)© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Quantization in TVM • Quantization within pre-quantized graph in TFLite or MxNet • Use high-level wrapper ops of QNN dialect© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. TVM Overview Framework Graph Mxnet TF ….

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Deploying GluonCV models using TVM© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon with TVM© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Deploy GluonCV Models https://arxiv.org/pdf/1907.02154.pdf© 2019, Amazon Web Services, Inc. or its Affiliates Amazon Trademark Overall Performance AWS DeepLens Acer aiSage NVIDIA Jetson Nano© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Effects of Vision-specific