Chapter 3 - Learning Techniques “The more that you read, the more things you will know. The more that you learn, the more places you'll go.” ― Dr. Seuss Model quality is an important benchmark to evaluate evaluate the performance of a deep learning model. A language translation application that uses a low quality model would struggle with consumer adoption because it wouldn’t serve its intended purpose translation accuracy would garner better consumer support. In this chapter, our focus will be on the techniques that enable us to achieve our quality goals. High quality models have an additional benefit in

Advanced Learning Techniques “Tell me and I forget, teach me and I may remember, involve me and I learn.” – Benjamin Franklin This chapter is a continuation of Chapter 3, where we introduced learning techniques techniques. To recap, learning techniques can help us meet our model quality goals. Techniques like distillation and data augmentation improve the model quality, without increasing the footprint of the model this chapter by presenting self-supervised learning which has been instrumental in the success of natural language models like BERT. Self-Supervised learning helps models to quickly achieve impressive

Compression Techniques “I have made this longer than usual because I have not had time to make it shorter.” Blaise Pascal In the last chapter, we discussed a few ideas to improve the deep learning efficiency efficiency. Now, we will elaborate on one of those ideas, the compression techniques. Compression techniques aim to reduce the model footprint (size, latency, memory etc.). We can reduce the model footprint gentle introduction to the idea of compression. Details of quantization and its applications in deep learning follow right after. The quantization section delves into the implementation details using code samples

Advanced Compression Techniques “The problem is that we attempt to solve the simplest questions cleverly, thereby rendering them unusually complex. One should seek the simple solution.” — Anton Pavlovich Pavlovich Chekhov In this chapter, we will discuss two advanced compression techniques. By ‘advanced’ we mean that these techniques are slightly more involved than quantization (as discussed in the second them, with an eye towards conceptual understanding as well as practically using them in your deep learning models. We start with sparsity. If your goal was to optimize your brain for storage, you can often

Machine Learning Lecture 10: Neural Networks and Deep Learning Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 2018 Deep Feedforward usually a highly non-linear function • Feedforward networks are of extreme importance to machine learning practioners • The conventional neural networks (CNN) used for object recognition from photos are units), and output layer 7 / 19 Neural Feedforward Networks (Contd.) • We approximate f ∗(x) by learning f(x) from the given training data • In the output layer, f(x) ≈ y for each training data, but the

Back to Basics: Debugging Techniques Bob Steagall CppCon 2021CppCon 2021 – Back to Basics: Debugging Techniques Copyright © 2021 Bob Steagall The Cost of Software Failures • January 2018, Tricentis’ salary -- $1.2T enterprise value lost for shareholders 2CppCon 2021 – Back to Basics: Debugging Techniques Copyright © 2021 Bob Steagall The Cost of Software Failures • Radiation overdoses from Therac-25 737 MAX MCAS system • System component design flaws 3CppCon 2021 – Back to Basics: Debugging Techniques Copyright © 2021 Bob Steagall Agenda • What are bugs? • What is debugging? • Challenges when

Machine Learning Pytorch Tutorial TA : 曾元（Yuan Tseng） 2022.02.18 Outline ● Background: Prerequisites & What is Pytorch? ● Training & Testing Neural Networks in Pytorch ● Dataset & Dataloader ● Tensors link3 2. Deep Learning Basics ■ Prof. Lee’s 1st & 2nd lecture videos from last year ■ ref: link1, link2 Some knowledge of NumPy will also be useful! What is PyTorch? ● An machine learning framework in computing with more cores for arithmetic calculations ○ See What is a GPU and do you need one in deep learning? Tensors – Gradient Calculation >>> x = torch.tensor([[1., 0.], [-1., 1.]], requires_grad=True)