Graph Target-independent Relay passes Target-optimized graph Target-dependent Relay passes Intel x86 ARM CPU Nvidia GPU ARM GPU Schedule templates written in TVM Tensor IR .. More targets AutoTVM QNN Dialect QNN passes Target-independent Relay passes Target-optimized Int8 Relay Graph Intel x86 schedule ARM CPU schedule Nvidia GPU schedule ARM GPU schedule Relay Int8 Graph Target-dependent Relay operators • QNN Optimization passes • Some optimizations are easier at QNN level • Intel x86 VNNI requires conv input dtypes to uint8 x int8© 2019, Amazon Web Services, Inc. or its Affiliates

CALLING C AND FORTRAN CODE 353 SIMD Values Note: This feature is currently implemented on 64-bit x86 and AArch64 platforms only. If a C/C++ routine has an argument or return value that is a native SIMD The tuple must be the same size as the SIMD type. For example, a tuple representing an __m128 on x86 must have a size of 16 bytes. • The element type of the tuple must be an instance of VecElement{T} 27. CALLING C AND FORTRAN CODE 365 C name Standard Julia Alias Julia Base Type char Cchar Int8 (x86, x86_64), UInt8 (powerpc, arm) long Clong Int (UNIX), Int32 (Windows) unsigned long Culong UInt (UNIX)

must be the same size and elements as the SIMD type. For example, a tuple representing an __m128 on x86 must have a size of 16 bytes and Float32 elements. • The element type of the tuple must be an instance 27. CALLING C AND FORTRAN CODE 365 C name Standard Julia Alias Julia Base Type char Cchar Int8 (x86, x86_64), UInt8 (powerpc, arm) long Clong Int (UNIX), Int32 (Windows) unsigned long Culong UInt (UNIX) following program, when compiled with julia -O3 generates two SIMD addition instructions (addps) on x86 systems: const m128 = NTuple{4,VecElement{Float32}} function add(a::m128, b::m128) (VecElement(a[1]

of T is not known statically. SIMD Values Note: This feature is currently implemented on 64-bit x86 and AArch64 platforms only.CHAPTER 28. CALLING C AND FORTRAN CODE 375 If a C/C++ routine has an argument The tuple must be the same size as the SIMD type. For example, a tuple representing an __m128 on x86 must have a size of 16 bytes. • The element type of the tuple must be an instance of VecElement{T} 28. CALLING C AND FORTRAN CODE 387 C name Standard Julia Alias Julia Base Type char Cchar Int8 (x86, x86_64), UInt8 (powerpc, arm) long Clong Int (UNIX), Int32 (Windows) unsigned long Culong UInt (UNIX)

must be the same size and elements as the SIMD type. For example, a tuple representing an __m128 on x86 must have a size of 16 bytes and Float32 elements. • The element type of the tuple must be an instance 28. CALLING C AND FORTRAN CODE 390 C name Standard Julia Alias Julia Base Type char Cchar Int8 (x86, x86_64), UInt8 (powerpc, arm) long Clong Int (UNIX), Int32 (Windows) unsigned long Culong UInt (UNIX) following program, when compiled with julia -O3 generates two SIMD addition instructions (addps) on x86 systems: const m128 = NTuple{4,VecElement{Float32}} function add(a::m128, b::m128) (VecElement(a[1]

must be the same size and elements as the SIMD type. For example, a tuple representing an __m128 on x86 must have a size of 16 bytes and Float32 elements. • The element type of the tuple must be an instance 28. CALLING C AND FORTRAN CODE 391 C name Standard Julia Alias Julia Base Type char Cchar Int8 (x86, x86_64), UInt8 (powerpc, arm) long Clong Int (UNIX), Int32 (Windows) unsigned long Culong UInt (UNIX) following program, when compiled with julia -O3 generates two SIMD addition instructions (addps) on x86 systems: const m128 = NTuple{4,VecElement{Float32}} function add(a::m128, b::m128) (VecElement(a[1]

must be the same size and elements as the SIMD type. For example, a tuple representing an __m128 on x86 must have a size of 16 bytes and Float32 elements. • The element type of the tuple must be an instance 28. CALLING C AND FORTRAN CODE 390 C name Standard Julia Alias Julia Base Type char Cchar Int8 (x86, x86_64), UInt8 (powerpc, arm) long Clong Int (UNIX), Int32 (Windows) unsigned long Culong UInt (UNIX) following program, when compiled with julia -O3 generates two SIMD addition instructions (addps) on x86 systems: const m128 = NTuple{4,VecElement{Float32}} function add(a::m128, b::m128) (VecElement(a[1]

must be the same size and elements as the SIMD type. For example, a tuple representing an __m128 on x86 must have a size of 16 bytes and Float32 elements. • The element type of the tuple must be an instance 28. CALLING C AND FORTRAN CODE 390 C name Standard Julia Alias Julia Base Type char Cchar Int8 (x86, x86_64), UInt8 (powerpc, arm) long Clong Int (UNIX), Int32 (Windows) unsigned long Culong UInt (UNIX) following program, when compiled with julia -O3 generates two SIMD addition instructions (addps) on x86 systems: const m128 = NTuple{4,VecElement{Float32}} function add(a::m128, b::m128) (VecElement(a[1]

must be the same size and elements as the SIMD type. For example, a tuple representing an __m128 on x86 must have a size of 16 bytes and Float32 elements. • The element type of the tuple must be an instance 28. CALLING C AND FORTRAN CODE 390 C name Standard Julia Alias Julia Base Type char Cchar Int8 (x86, x86_64), UInt8 (powerpc, arm) long Clong Int (UNIX), Int32 (Windows) unsigned long Culong UInt (UNIX) following program, when compiled with julia -O3 generates two SIMD addition instructions (addps) on x86 systems: const m128 = NTuple{4,VecElement{Float32}} function add(a::m128, b::m128) (VecElement(a[1]

must be the same size and elements as the SIMD type. For example, a tuple representing an __m128 on x86 must have a size of 16 bytes and Float32 elements. • The element type of the tuple must be an instance 28. CALLING C AND FORTRAN CODE 387 C name Standard Julia Alias Julia Base Type char Cchar Int8 (x86, x86_64), UInt8 (powerpc, arm) long Clong Int (UNIX), Int32 (Windows) unsigned long Culong UInt (UNIX) following program, when compiled with julia -O3 generates two SIMD addition instructions (addps) on x86 systems: const m128 = NTuple{4,VecElement{Float32}} function add(a::m128, b::m128) (VecElement(a[1]