that enable us to achieve our quality goals. High quality models have an additional benefit in footprint constrained environments like mobile and edge devices where they provide the flexibility to trade learning techniques. It is followed by a short discussion on exchanging model quality and model footprint. An in-depth discussion of data augmentation and distillation follows right after. Following the and/or label efficient training setup, can we exchange some of this to achieve a model with a better footprint? The next subsection elaborates it further. Using learning techniques to build smaller and faster

without increasing the footprint of the model (size, latency, etc). And as we have described earlier, some of these improved quality metrics can be traded off for a smaller footprint as desired. Continuing techniques to help you improve your model’s quality metrics without taking a hit on any of the footprint metrics. These techniques might get superseded by other better methods over time, but again our Proceedings of the 26th Annual International Conference on Machine Learning." Curriculum learning. Association for Computing Machinery, 14 June 2009, pp. 41-48, doi:10.1145/1553374.1553380. 18 Allgower, Eugene

time could still turn out to be expensive. There is also a very real concern around the carbon footprint of datacenters that are used for training and deploying these large models. Large organizations about. First, we have quality metrics like accuracy, precision, recall, F1, AUC, etc. Then we have footprint metrics like model size, latency, RAM, etc. Empirically, we have seen that larger deep learning train and deploy hence worse footprint. On the other hand, smaller and shallower models might have suboptimal quality. Figure 1-6: Trade-offs between quality metrics and footprint metrics. In case we have

changes to add a couple of windows and a balcony. Similarly, to gain orders of magnitude in terms of footprint or quality, we should consider employing suitable efficient architectures. The progress of deep Efficient Architectures aim to improve model deployability by proposing novel ways to reduce model footprint and improve inference efficiency while preserving the problem solving capabilities of their giant 2. Even after compression, the vocabulary itself is large: Large vocabularies have a tangible footprint by themselves, which excludes the actual embeddings. They are persisted with the model to help with

compression techniques. Compression techniques aim to reduce the model footprint (size, latency, memory etc.). We can reduce the model footprint by reducing the number of trainable parameters. However, this approach categories: footprint metrics such as model size, prediction latency, RAM consumption and the quality metrics, such as accuracy, F1, precision and recall as shown in table 2-1. Footprint Metrics Quality ● Accuracy ● Precision ● Recall ● F1 ● AUC Table 2-1: A few examples of footprint and quality metrics. The footprint and the quality metrics are typically at odds with each other. As stated earlier

precision. We will leave the biases untouched since they do not contribute significantly to the layer’s footprint. num_bits = 8 weights_dequantized, weights_reconstruction_error_quant = simulate_quantization( their specific model training setup. Sparsity by itself helps with compressing the model size (footprint metric) since many connections can be removed without a noticeable impact on quality metrics. However compression technique, yet implementing it is quite straightforward. We can achieve quality and footprint gains on top of quantization because clustering is a much more generic approach of allocating precision

定层的数据作为输入，跨多个后续层 对数据进行处理，然后将数据发送到下一个GPU。与单个GPU所能处理的数据相比，我们可以用更大的网络 处理数据。此外，每个GPU占用的显存（memory footprint）可以得到很好的控制，虽然它只是整个网络显 存的一小部分。 然而，GPU的接口之间需要的密集同步可能是很难办的，特别是层之间计算的工作负载不能正确匹配的时候， 还有层之间的接口需要大量的 , & Mikolov, T. (2017). Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5, 135– 146. 769 [Bollobas, 1999] Bollobás, B. (1999). Linear Learning word vectors for sentiment analysis. Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies-volume 1 (pp. 142–150). [McCann et

turning the knobs (the hyperparameters) until we are satisfied with the sound (model quality and footprint) that each string produces. Unlike the guitar which has a few knobs, the hyperparameter search space

如何将教室里的学生按爱好、身高划分为5类？ ✓ 降维（ Dimensionality Reduction ） ✓ 如何将将原高维空间中的数据点映射到低维度的空间中？ ✓ 关联规则（ Association Rules） ✓ 很多买尿布的男顾客，同时买了啤酒，可以从中找出什么规律来提 高超市销售额？ ✓ 推荐系统（ Recommender systems） ✓ 很多客户经常上网购物，根据他们的浏览商品的习惯，给他们推荐

FP-Growth算法 3 1.关联规则概述 01 关联规则概述 02 Apriori 算法 03 FP-Growth算法 4 1.关联规则概述 关联规则 关联规则（Association Rules）反映一个事物与其他事物之间的相互依存 性和关联性。如果两个或者多个事物之间存在一定的关联关系，那么，其中 一个事物就能够通过其他事物预测到。 关联规则可以看作是一种IF-THEN关系。假设