performance tradeoff. Next, the chapter goes over weight sharing using clustering. Weight sharing, and in particular clustering is a generalization of quantization. If you noticed, quantization ensures range. It creates equal sized quantization ranges (bins), regardless of the frequency of data. Clustering helps solve that problem by adapting the allocation of precision to match the distribution of the started to combine these two forms to achieve both accuracy and latency gains. Weight Sharing using Clustering Recall that in quantization, we divided the original floating point domain between and into non-overlapping

/ 46 Outline 1 Clustering 2 K-Means Method 3 K-Means Optimization Problem 4 Kernel K-Means 5 Hierarchical Clustering Feng Li (SDU) K-Means December 28, 2021 2 / 46 Clustering Usually an unsupervised labels A good clustering is one that achieves: High within-cluster similarity Low inter-cluster similarity Feng Li (SDU) K-Means December 28, 2021 3 / 46 Similarity can be Subjective Clustering only looks important in clustering Also important to define/ask: “Clustering based on what”? Feng Li (SDU) K-Means December 28, 2021 4 / 46 Clustering: Some Examples Document/Image/Webpage Clustering Image Segmentation

/// Recompute the centroids given the current clustering fn recompute_centroids( xs: &[Vec<$kind>], clustering: &[usize], k: usize, ndims]; let mut n_cluster: $kind = 0.0; xs.iter().zip(clustering.iter()).for_each(|(xi, &zi)| { if zi == cluster_ix { } /// Assign the N D-dimensional data, `xs`, to `k` clusters using K- Means clustering pub fn kmeans(xs: Vec>, k: usize) -> Vec { assert

Discovering Clusters Feng Li (SDU) Overview September 6, 2023 29 / 57 Unsupervised Learning: Clustering (Contd.) Feng Li (SDU) Overview September 6, 2023 30 / 57 Unsupervised Learning: Discovering required compared to supervised learning. At the same time, improving the results of unsupervised clustering to the expectations of the user. With lots of unlabeled data the decision boundary becomes apparent Constrained Clustering Distance Metric Learning Manifold based Learning Sparsity based Learning (Compressed Sensing) Feng Li (SDU) Overview September 6, 2023 40 / 57 Constrained Clustering When we have

. . . . . . . . . . . . . . . . . . . . . . . . 295 43.8. Changing the username/password for clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295 43.9. Securing non-destructive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332 50.4. Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 500 87.12. Clustering with JGroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . 291 42.8. Changing the username/password for clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291 42.9. Securing non-destructive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328 49.4. Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499 86.13. Clustering with JGroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . 291 42.8. Changing the username/password for clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291 42.9. Securing non-destructive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328 49.4. Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499 86.13. Clustering with JGroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . 291 42.8. Changing the username/password for clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291 42.9. Securing non-destructive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328 49.4. Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498 86.13. Clustering with JGroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . 306 43.8. Changing the username/password for clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306 43.9. Securing non-destructive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342 50.4. Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 511 87.12. Clustering with JGroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . 306 43.8. Changing the username/password for clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306 43.9. Securing non-destructive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342 50.4. Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 511 87.12. Clustering with JGroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .