10/22 4/25 800MM Big Six* USA Technology Company CapEx *Apple, NVIDIA, Microsoft, Alphabet, Amazon (AWS only), & Meta Platforms Source: Capital IQ (3/25), Morgan Stanley 2014 2024 CapEx, $B +63% $212B on internet infrastructure, which allows for wicked-fast adoption of easy-to-use broad-interest services. AI Technology Compounding = Numbers Behind The Momentum22 Knowledge Distribution Evolution = free humanity from menial repetitive work and to dramatically increase the production of goods and services could presage an era of peace and plenty. The capacity to accelerate scientific research could

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Deploying GluonCV models using TVM© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon TVM© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Deploy GluonCV Models https://arxiv.org/pdf/1907.02154.pdf© 2019, Amazon Web Services, Inc. or its Affiliates Affiliates. All rights reserved. Amazon Trademark Overall Performance AWS DeepLens Acer aiSage NVIDIA Jetson Nano© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark

Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon/Intel Confidentia Presenter: Zhi Chen, Cody Yu Amazon SageMaker Neo, Deep Engine Science Bring Your Own Codegen to TVM AWS AI© 2019 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Considering You... Design and manufacture a deep learning chip which achieves amazing performance on widely-used operators (e.g Suppression (NMS) is too new to be supported by your chip But NMS is supported by TVM!© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Let TVM Be the Compiler of Your Chip Your chip can

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Animesh Jain Amazon SageMaker Neo Compilation of Quantized Models in TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates 𝑟𝑒𝑎𝑙_𝑣𝑎𝑙𝑢𝑒 = 𝑠𝑐𝑎𝑙𝑒 ∗ (𝑞𝑢𝑎𝑛𝑡𝑖𝑧𝑒𝑑_𝑣𝑎𝑙𝑢𝑒 − 𝑧𝑒𝑟𝑜_𝑝𝑜𝑖𝑛𝑡)© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Quantization in TVM • Quantization within TVM - pre-quantized graph in TFLite or MxNet • Use high-level wrapper ops of QNN dialect© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. TVM Overview Framework Graph Mxnet TF …. parsers

Amazon Web Services, Inc. or its Affiliates. All rights reserved. Presenter: Haichen Shen, Yao Wang Amazon SageMaker Neo, Deep Engine Science Dynamic Model in TVM AWS AI© 2019, Amazon Web Services, Inc. loop Limitation of TVM/graph runtime ● Cannot compile and run dynamic models© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Support dynamic model in TVM ● Support Any-dim in dispatch for a (sub-)graph In collaboration with Jared Roesch, Zhi Chen, Wei Chen© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Any” in Relay typing Any: represent an unknown

azure.com 671B（全量模型） 需注册微软账户并创建订阅，免费部署，支持参数调节。 亚马逊AWS https://aws.amazon.com/c n/blogs/aws/deepseek-r1- models-now-available-on- aws 671B（全量模型） 需注册AWS账户，填写付款方式，免费部署。 Cerebras https://cerebras.ai 70B

to TVM o_uTVM: support for microcontrollers in TVM o_ Virtual Machine and dynamic NNs support (w/ AWS folks) o_ Improved NLP support, with focus on transformers QQ octoML Core Infrastructure Refactors currently implemented using copy, 10 Virtual Machine e Many improvements from contributors at UW, AWS, and OctoML. e Initial implementation is quickly moving towards production quality. o _VM compiler Apache(incubating) community members. e ASF Mentors and PMC members who make this awesome project Possiblel ee AWS for hosting the first Bay Area meetup QQ octoML 14 Annual TVM Conference 2019 Organized and participated

evals How Morgan Stanley iterated to ensure quality and safety As a global leader in financial services, Morgan Stanley is a relationship business. Not surprisingly, there were some questions across Generative AI Solutions To find out more, watch Morgan Stanley: Shaping the Future of Financial Services and ask us about our Eval Frameworks. 7 AI in the EnterpriseEvals defined Evaluation is the process