the first chapter, we briefly introduced learning techniques such as regularization, dropout, data augmentation, and distillation to improve quality. These techniques can boost metrics like accuracy, precision recall, etc. which often are our primary quality concerns. We have chosen two of them, namely data augmentation and distillation, to discuss in this chapter. This is because, firstly, regularization and dropout dropout are fairly straight-forward to enable in any modern deep learning framework. Secondly, data augmentation and distillation can bring significant efficiency gains during the training phase, which is

the training pipeline like data augmentation, layer and channel configurations can also be parameterized using hyperparameters. For example, when using image data augmentation with rotation, we can treat also have additional parameters which could be searched as well. transformation parameters in data augmentation layer contribute to performance improvements while others like learning rate, batch size or layers.Dense(size, activation='relu'), layers.Dense(5, activation='softmax') ]) Our model, input data and the hyperparameter trial set is ready. Let's go ahead and train the model, each time choosing

learning techniques can help us meet our model quality goals. Techniques like distillation and data augmentation improve the model quality, without increasing the footprint of the model (size, latency, etc) As we described in chapter 3’s ‘Learning Techniques and Efficiency’ section, labeling of training data is an expensive undertaking. Factoring in the costs of training human labelers on a given task, and significantly improve the quality you can achieve while retaining the same labeling costs i.e., training data-efficient (specifically, label efficient) models. We will describe the general principles of Self-Supervised

features in the input. Recurrent Neural Nets (RNNs) facilitated learning from the sequences and temporal data. These breakthroughs contributed to bigger and bigger models. Although they improved the quality of you see, books you read, food you enjoy and so on), without the need of knowing all the encyclopedic data about them. When working with deep learning models and inputs such as text, which are not in numerical high-dimensional data into low-dimension, while retaining the properties from the high-dimensional representation. It is useful because it is often computationally infeasible to work with data that has a large

there might not be a single algorithm that works perfectly, and there is a large amount of unseen data that the algorithm needs to process. Unlike traditional algorithm problems where we expect exact optimal certainty the exact content that you would end up clicking on, at that particular moment, with more data and sophisticated algorithms, these models can be trained to be fairly accurate over a longer term Availability of labelled data Even if one has enough compute, and sophisticated algorithms, solving classical machine learning problems relies on the presence of sufficient labeled data. With deep learning

数据增强 主讲人：龙良曲 Big Data ▪ The key to prevent Overfitting Sample more data? Limited Data ▪ Small network capacity ▪ Regularization ▪ Data argumentation Recap Data argumentation ▪ Flip ▪ Rotate com/nanonets/how-to-use-deep-learning-when-you-have-limited-data- part-2-data-augmentation-c26971dc8ced Flip Rotate Rotate Scale Crop Part Noise ▪ Data argumentation will help ▪ But not too much 下一课时

0.01 的高斯分布： ? = 1. ? + . + ?, ? ∼ ?( , . 12) 通过随机采样? = 1 次，可以获得?个样本的训练数据集?train，代码如下： data = []# 保存样本集的列表 for i in range(100): # 循环采样 100 个点 x = np.random.uniform(-10., 10.) # 随机采样输入 random.normal(0., 0.01) # 得到模型的输出 y = 1.477 * x + 0.089 + eps data.append([x, y]) # 保存样本点 data = np.array(data) # 转换为 2D Numpy 数组 通过 for 循环进行 100 次采样，每次从均匀分布?(−1 ,1 )中随机采样一个数据?，同时从 均值为 1000 次，返回最优 w*,b*和训练 Loss 的下降过程 [b, w]= gradient_descent(data, initial_b, initial_w, lr, num_iterations) loss = mse(b, w, data) # 计算最优数值解 w,b 上的均方差 print(f'Final loss:{loss}, w:{w}, b:{b}')

of algorithms, inspired from the model of human brain. Deep learning is becoming more popular in data science fields like robotics, artificial intelligence(AI), audio & video recognition and image recognition -U scikit-learn Seaborn Seaborn is an amazing library that allows you to easily visualize your data. Use the below command to install: pip install seaborn You could see the message similar as specified json { "image_data_format": "channels_last", "epsilon": 1e-07, "floatx": "float32", "backend": "tensorflow" } Here,  image_data_format represent the data format.  epsilon

metrics=['accuracy']) # 生成虚拟数据 import numpy as np data = np.random.random((1000, 100)) labels = np.random.randint(2, size=(1000, 1)) # 训练模型，以 32 个样本为一个 batch 进行迭代 model.fit(data, labels, epochs=10, batch_size=32) compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy']) # 生成虚拟数据 import numpy as np data = np.random.random((1000, 100)) labels = np.random.randint(10, size=(1000, 1)) # 将标签转换为分类的 one-hot one_hot_labels = keras.utils.to_categorical(labels, num_classes=10) # 训练模型，以 32 个样本为一个 batch 进行迭代 model.fit(data, one_hot_labels, epochs=10, batch_size=32) 3.1.5 例子 这里有几个可以帮助你开始的例子！ 在 examples 目录 中，你可以找到真实数据集的示例模型：

tensor 理解为是“data”。 我们需要先导入 pytorch，顺便导入 numpy： import torch import numpy as np 现在我们尝试将 list 或者 np.array 转换为 pytorch 的数组： data1 = [ [ 1 , 2 ] , [ 3 , 4 ] ] data_tensor = torch . tensor ( data1 ) print print ( data_tensor . shape ) np_array1 = np . array ( data1 ) data_tensor = torch . from_numpy( np_array1 ) print ( data_tensor . shape ) 输出都是： torch . Size ( [ 2 , 2 ] ) 对于二维 tensor 之间的相乘，@ 和 .matmul 7 y = data_tensor @ data_tensor .T print (y) y = data_tensor ∗ data_tensor print (y) 输出分别是： [ [ 5 , 11] , [11 , 2 5 ] ] [ [ 5 , 11] , [11 , 2 5 ] ] tensor 可以转化为 numpy： np_array2 = data_tensor