variable depth child networks. Figure 7-4 shows a sketch of their search procedure. It involves a controller which samples the search space to generate candidate architectures. The candidates are used as to the controller as reward signals. The controller incorporates the rewards signals in its gradient updates. Zoph et. al. modeled NAS as a reinforcement learning (RL) problem where the controller is a recurrent networks are the players whose rewards are determined by their performance on the target dataset. The controller model learns to generate better architectures as the search game progresses. Figure 7-4: An overview

The controller can be thought of as a unit that generates candidate models. These candidate models are evaluated and is used to update the state, and generate better candidate models A controller unit needs to be optimized (accuracy, precision, recall, etc.), and the feedback is passed back to the controller to make better suggestions in the future. NAS has been used to generate State of the Art networks

reshape 函数之外，还有另外一个基于 tensor 的维度 转换方法 tensor.view(), 它的用法代码演示如下： x = torch.randn(4, 4) print(x.size()) x = x.view(-1, 8) print(x.size()) x = x.view(1, 1, 4, 4) print(x.size()) 运行结果如下： torch.Size([4 Size([2, 8]) torch.Size([1, 1, 4, 4]) 其中 torch.randn(4, 4) 是创建一个 4x4 的随机张量；x.view(-1, 8) 表示转换为每行八列的，-1 表示自动计算行数；x.view(1, 1, 4, 4) 表示转换为 1x1x4x4 的四维张量。其中 torch.size 表示 输出数组维度大小。 ● 其它属性操作 通道交换与寻找最大值是 print(x.size()) x = torch.tensor([2., 3., 4.,12., 3., 5., 8., 1.]) print(torch.argmax(x)) x = x.view(-1, 4) print(x.argmax(1)) 运行结果如下： torch.Size([5, 5, 3]) torch.Size([3, 5, 5]) tensor(3) tensor([3

relu(self.conv1(x)), 2) x = F.max_pool2d(F.relu(self.conv2(x)), 2) # 改变数据的维度 x = x.view(-1, self.num_flat_features(x)) x = F.relu(self.fc1(x)) x = F.relu(self.fc2(x)) MSELoss是一个比较简单的损失函数，它计算输出和目标间的均方误差， 例如： output = net(input) target = torch.rand(10) target = target.view(1, -1) criterion = nn.MSELoss() loss = criterion(output, target) print(loss) tensor(0.4526, gra .grad_fn 属性，将看到如下所示的计算图。 input -> conv2d -> relu -> maxpool2d -> conv2d -> relu -> maxpool2d -> view -> linear -> relu -> linear -> relu -> linear -> MSELoss -> loss 所以，当我们调用 loss.backward() 时,整张计算图都会

ndim x.dim() x.size x.nelement() 形状操作 x.reshape x.reshape(相当于 tensor.contiguous().view()); x.view x.flatten x.view(-1)；nn Flatten() 类型转换 np.floor(x) torch.floor(x); x.floor() 比较 np.less x.lt np.less_equal/np

编码函数，depth 设置向量长度 out = torch.zeros(label.size(0), depth) idx = torch.LongTensor(label).view(-1, 1) out.scatter_(dim=1, index=idx, value=1) return out y = torch.tensor([0,1,2 # x: [b, 1, 28, 28], y: [512] # 打平操作：[b, 1, 28, 28] => [b, 784] x = x.view(x.size(0), 28*28) # 送入网络模型，=> [b, 10] out = model(x) # 标签进行 one-hot transpose 操作、复制数据 tile 操作等，下面将一一 介绍。 4.7.1 改变视图 在介绍改变视图 reshape 操作之前，先来认识一下张量的存储(Storage)和视图(View)的 概念。张量的视图就是人们理解张量的方式，比如 shape 为[2,3,4,4]的张量?，从逻辑上可 以理解为 2 张图片，每张图片 4 行 4 列，每个位置有 RGB 3 个通道的数据；张量的存储体

Tensor维度变换 主讲人：龙良曲 Operation ▪ View/reshape ▪ Squeeze/unsqueeze ▪ Transpose/t/permute ▪ Expand/repeat 2 View reshape ▪ Lost dim information 3 Flexible but prone to corrupt 4 Squeeze v.s. unsqueeze

we tolerate? Let us slowly build up to that by exploring how quantization can help us. A Generic View of Quantization Quantization is a common compression technique that has been used across different matrix. D is often a one-dimension vector, hence the addition is cheap both from the latency point of view and size wise (since C dominates the size). In fact, the general formulation of Y = XW + b, is the

should get a figure similar to Fig. 2. If you are using Matlab/Octave, you can use the orbit tool to view this plot from different viewpoints. What is the relationship between this 3D surface and the value

Tensors – PyTorch v.s. NumPy ● Many functions have the same names as well PyTorch NumPy x.reshape / x.view x.reshape x.squeeze() x.squeeze() x.unsqueeze(1) np.expand_dims(x, 1) ref: https://github.com/