conv2d_strategy_cpu(attrs, inputs, out_type, target): strategy = OpStrategy() layout = attrs.data_layout if layout == "NCHW": oc, ic, kh, kw = inputs[1].shape strategy.register_spec elif layout == "NHWC": strategy.register_default_implement(wrap_compute_conv2d(topi.nn.conv2d_nhwc), topi.x86.schedule_conv2d_nhwc) elif layout == "NCHWc": conv2d_NCHWc. Graph tuning is well defined for each subgraph. 3. Avoid runtime layout tracking system for operator requires layout transformation to optimize.© 2019, Amazon Web Services, Inc. or its Affiliates

inputs, lambda ins, outs: tvm.call_packed('tvm.accel.accel_fused', attrs['path'], attrs['output_layout'], attrs['model_name'], outs[0], *ins ), name=name) return out >> 10© Copyright 2018 Xilinx Example accel.accel_fused") def accel_fused(graph_path, output_layout, out, *ins ): path = c_char_p(graph_path.value).value layout = c_char_p(output_layout.value).value … >> 12© Copyright 2018 Xilinx Performance

ARM CPU schedule Nvidia GPU schedule ARM GPU schedule Relay Int8 Graph Target-dependent Relay layout opt© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Outline • QNN Dialect ARM CPU schedule Nvidia GPU schedule ARM GPU schedule Relay Int8 Graph Target-dependent Relay layout opt© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Quantization Approaches ARM CPU schedule Nvidia GPU schedule ARM GPU schedule Relay Int8 Graph Target-dependent Relay layout opt© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Conclusion • TVM community

驱动万物智能 Alios TVM @ ARM CPU INT8 * Cache 芍四 Data FO Data FOData … QNNPACK Convolution 。，NHWC layout Cach， 浆百 FeU Cach- 区下 。， im2col + pack re 。 Tensorize Convolution Workload Performance AiOS 1驱动万物智能 Alios TVM @ ARM CPU INT8 Depthwise Convolution 。， NHWC layout 。 Using TVM schedule primitive completely, no tensorize 。 Some Experience: 1 Avoid DataPack 2 国QNNPACK 1core 四QNNPACK4core 四TVM1core 四TVM4core AiOS 1驱动万物智能 Alios TVM @ ARM CPU FP32 。，NHWC layout 。 For pointwise convolution we implement im2col schedule 。 No tensorize, but in schedule to cooperate

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1067 55.4 DataType layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1067 55.5 Function methods types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1543 101.4 Memory layout of Julia Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1552 101.5 Eval of Julia DataType A DataType may be abstract or concrete. If it is concrete, it has a specified size, storage layout, and (optionally) field names. Thus a primitive type is a DataType with nonzero size, but no field

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1067 55.4 DataType layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1067 55.5 Function methods types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1543 101.4 Memory layout of Julia Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1552 101.5 Eval of Julia DataType A DataType may be abstract or concrete. If it is concrete, it has a specified size, storage layout, and (optionally) field names. Thus a primitive type is a DataType with nonzero size, but no field

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1248 57.4 DataType layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1248 57.5 Function methods types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1823 105.4 Memory layout of Julia Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1832 105.5 Eval of Julia 137 A DataType may be abstract or concrete. If it is concrete, it has a specified size, storage layout, and (optionally) field names. Thus a primitive type is a DataType with nonzero size, but no field

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1248 57.4 DataType layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1248 57.5 Function methods types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1823 105.4 Memory layout of Julia Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1832 105.5 Eval of Julia 137 A DataType may be abstract or concrete. If it is concrete, it has a specified size, storage layout, and (optionally) field names. Thus a primitive type is a DataType with nonzero size, but no field

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1248 57.4 DataType layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1248 57.5 Function methods types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1823 105.4 Memory layout of Julia Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1832 105.5 Eval of Julia 137 A DataType may be abstract or concrete. If it is concrete, it has a specified size, storage layout, and (optionally) field names. Thus a primitive type is a DataType with nonzero size, but no field

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1301 58.4 DataType layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1301 58.5 Function methods types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1876 106.4 Memory layout of Julia Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1885 106.5 Eval of Julia 137 A DataType may be abstract or concrete. If it is concrete, it has a specified size, storage layout, and (optionally) field names. Thus a primitive type is a DataType with nonzero size, but no field