specifics X78Structured and Unstructured Sparsity - Lots of 'free' wins from exploring sparsity in modern ML models - Can often prune models to 80%+ sparsity(with retraining) - Massive speedups combined

Synthetic Data  Generate data with specific data characteristics  Systematic evaluation w/ datasize, sparsity, etc  Inappropriate for certain topics: compression, ML accuracy  “Real” Data Repositories  0 [J. Sommer, M. Boehm, A. V. Evfimievski, B. Reinwald, P. J. Haas: MNC: Structure- Exploiting Sparsity Estimation for Matrix Expressions. SIGMOD 2019] 15 706.015 Introduction to Scientific Writing #Guidelines] [J. Sommer, M. Boehm, A. V. Evfimievski, B. Reinwald, P. J. Haas: MNC: Structure- Exploiting Sparsity Estimation for Matrix Expressions. SIGMOD 2019] 27 706.015 Introduction to Scientific Writing

N. Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. CoRR, abs/2101.03961, 2021. URL https://arxiv.org/ abs/2101.03961. L. Gao, S. Biderman, S. Black