we track: Latency & Throughput ˃ ML pipeline contains multiple stages, performance limited by slowest one ˃ Performance results based on Xilinx own runtime pipeline available in github (https://github es/mp_classify.py) Streamlined multi-process pipeline using shared memory Usually need >4 Pre-Process cores running to keep up with FPGA ˃ TVM pipeline needed. CPU/FPGA partitions ideally run in parallel Post-Process (fc/softmax/nms) FPGA Acceleration Pre-Process (resize)© Copyright 2018 Xilinx FPGA Pipeline report in MLSuite 1.5 (animated gif of ResNet-50, view in slideshow mode) >> 14© Copyright 2018

training. We set the maximum sequence length to 4K, and train DeepSeek-V2 on 8.1T tokens. We leverage pipeline parallelism to deploy different layers of a model on different devices, and for each layer, the light-weight training framework developed internally by our engineers. It employs a 16-way zero-bubble pipeline parallelism (Qi et al., 2023), an 8-way expert parallelism (Lepikhin et al., 2021), and ZeRO-1 data models. arXiv preprint arXiv:2309.00071, 2023. P. Qi, X. Wan, G. Huang, and M. Lin. Zero bubble pipeline parallelism. arXiv preprint arXiv:2401.10241, 2023. S. Rajbhandari, J. Rasley, O. Ruwase, and

new/tuned TVM schedules using fast Integer operations like Intel VNNI, ARM Dot, Nvidia DP4A • Full pipeline is available. Please try it and give suggestions. • Open-source discussions formed the foundations

libtvm_hexagon_runtime.so Alios TVM @ Hexagon DSP 。 Compute Kernel Offload to DSP ，loop nests marked as pipeline 。， Implement complete Hexagon runtime based on community PR. ADSPRPC Framework Applications Processor