损失函数。这里我们使用 3.1节中描述的平方损失函数。 在实现中，我们需要将真实值y的形状转换为和预测值y_hat的形状相同。 def squared_loss(y_hat, y): #@save """均方损失""" return (y_hat - y.reshape(y_hat.shape)) ** 2 / 2 3.2.6 定义优化算法 正如我们在 3.1节中讨论的，线性回归有解析 下面，我们创建一个数据样本y_hat，其中包含2个样本 在3个类别的预测概率，以及它们对应的标签y。有了y，我们知道在第一个样本中，第一类是正确的预测；而 在第二个样本中，第三类是正确的预测。然后使用y作为y_hat中概率的索引，我们选择第一个样本中第一个 类的概率和第二个样本中第三个类的概率。 y = torch.tensor([0, 2]) y_hat = torch.tensor([[0 3, 0.2, 0.5]]) y_hat[[0, 1], y] tensor([0.1000, 0.5000]) 现在我们只需一行代码就可以实现交叉熵损失函数。 def cross_entropy(y_hat, y): return - torch.log(y_hat[range(len(y_hat)), y]) cross_entropy(y_hat, y) 3.6. softmax回归的从零开始实现

h = self.encoder(x) # 隐向量 h 经过解码器映射为重建的 x_hat 张量 # [b, 10] => [b, 784] x_hat = self.decoder(h) return x_hat 12.2.5 网络训练 自编码器的训练过程与分类器的基本一致，通过误差函数计算出重建向量 logits = model(x.reshape([-1, 784])) x_hat = torch.sigmoid(logits) # 恢复为 28x28,[b, 784] => [b, 28, 28] x_hat = x_hat.reshape([-1, 28, 28]) # 输入的前 50 张+重建的前 50 张图片合并，[b x.squeeze(dim=1) # [b,1,28,28]=>[b,28,28] x_concat = torch.cat([x[:50], x_hat[:50]], dim=0) # x_concat = x_hat 预览版202112 第 12 章 自编码器 8 x_concat = x_concat.cpu().numpy() * 255

search trials with two hyperparameters and . The blue contours mark the positive results while the red ones indicate the trials with high losses. The density of trials is identical in both the regions which algorithms for two hyperparameters. The blue contours show the regions with positive results while the red contours indicate poor results. The 'x' marks indicate the trials. The images are sourced under CC across the search space. Even though the trials in the blue region perform better than the ones in the red region, the search algorithm makes no effort to search "more" in the blue region. In other words, it

this image, each pixel is represented as three 8-bit numbers (ranging from 0 to 255) that specify red, green and blue intensity values. Our bird photo contains thousands of colors, but we’d like to reduce three-dimensional matrix A whose first two indices identify a pixel position and whose last index represents red, green, or blue. For example, A(50, 33, 3) gives you the blue intensity of the pixel at position y

pareto-optimal models that simply cost less resources to train and/or deploy. This means going from the red dots in Figure 3 to the green dots on the pareto-frontier. Having such a toolbox to make our models constructed based on past results. Figure 1-12: Bayesian Optimization over two dimensions x1 and x2. Red contour lines denote a high loss value, and blue contour lines denote a low loss value. The contours

space (i.e., n = 2). In the 3D space, the hypothesis function is represented by a hyperplane. The red points denote the training data, and the distance from the (read) training data to the hyperplane is

September 6, 2023 52 / 57 Example of Complexity Plots of polynomials having various degree, shown as red curves. x t M = 0 0 1 −1 0 1 Figure: Degree = 0 x t M = 1 0 1 −1 0 1 Figure: Degree

set_xlabel ( ’ count ’ ) ax . set_ylabel ( ’ cor rect (%) ’ ) plt . plot ( count , correctCurve , color=’ red ’ , linewidth =2.0 , l i n e s t y l e=’− ’ ) plt . show () 我们可以得到结果（我训练了很多次，有时候训练 1000 轮以后的正确率只有