Models in TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Quantization Overview • Represent FP32 numbers with a lower-precision INT8 numbers • Integer number stands All rights reserved. Quantization in TVM • Quantization within TVM - Automatic Quantization • TVM stack ingests a FP32 graph and a small dataset • Finds suitable quantization scale • Produces a quantized its Affiliates. All rights reserved. Quantization Appraoches in TVM Framework FP32 Graph MXNet Parser TF parser …. Relay FP32 Graph Relay Automatic Quantization Relay Int8 Graph Framework Pre-quantized

/c Weight Adjustment 和 90% 而 Baseline 国 INT8 quantization w/o WA 忻 INT8 quantization w/ WA 80% 70% 60% 50%6 MobileNet v1 MobileNet v1 0.5

2 : HIFI4 DSP PART 3 : _ PowervVR GPU [和| Alibaba AL.Labs 阿里巴巴人工智能实验室 ARM 32 CPU Resolution Quantization Orize Kernel ALIOS TVM Alibaba Al.Labs 阿里巴巴人工智能实验室 7= 590一I) rm 一 下Er (mm) =肪+2mxaM0 [5 (全-2)+o|

MLSuite 1.5 (animated gif of ResNet-50, view in slideshow mode) >> 14© Copyright 2018 Xilinx Quantization Tool – vai_q ˃ 4 commands in vai_q quantize ‒ Quantize network test ‒ Test network accuracy