child networks predicted by NASNet for CIFAR-10 and ImageNet approached or exceeded the contemporary state of the art models. These child networks were smaller and more efficient than the human designed models This process is repeated for cycles. There are two main mutations used in the mutation step: hidden state mutation and the operation mutation. One of these mutations is selected at random in each cycle. A (normal or reduction) as a mutation candidate. For hidden state mutation, a hidden state is picked at random and is replaced with another hidden state in the cell such that no cycles are created. For operation

is an end-to-end neural acoustic model for automatic speech recognition (ASR) that provides near state-of-the-art results on LibriSpeech among end-to-end ASR models without external data. This model script Representations from Transformers (BERT) is a new method of pretraining language representations which obtains state-of-the-art results on a wide array of Natural Language Processing (NLP) tasks. This model is based is an end-to-end neural acoustic model for automatic speech recognition (ASR) that provides near state-of-the-art results on LibriSpeech among end-to-end ASR models without external data. This model script

要保存和恢复模 型了。 model.state_dict() 函数可以得到模型的状态字典，里面包含了模型的参数权重与 bias 等信 息，我们可以用下面的代码来保存和恢复模型： # 保 存 模 型 torch . save ( model . state_dict () , path ) # 恢 复 模 型 model . load_state_dict ( torch . load 和优化器的类型，这时我们可以生成一个状态字典： # 保 存 模 型 state = { ’ model ’ : model . state_dict () , ’ optimizer ’ : optimizer . state_dict () , ’ epoch ’ : epoch} torch . save ( state , path ) # 恢 复 模 型 checkpoint = torch torch . load ( path ) model . load_state_dict ( checkpoint [ ’ model ’ ] ) optimizer . load_state_dict ( checkpoint [ ’ optimizer ’ ] ) epoch = checkpoint [ ’ epoch ’ ] 15 16 3.2. 初始化网络权重-方法一 现在

the input x Model (example) • Example: • Properties: • model = ManualLinearRegression() • model.state_dic() - returns a dictionary of trainable parameters with their current values • model.parameters() Only inference/evaluation – save only state_dict • Save: • torch.save(model.state_dict(), PATH) • Load: • model = TheModelClass(*args, **kwargs) • model.load_state_dict(torch.load(PATH)) • model.eval() training / inference • Save: • torch.save({ 'epoch': epoch, 'model_state_dict': model.state_dict(), 'optimizer_state_dict': optimizer.state_dict(), 'loss': loss, ... }, PATH) • Load: • model = TheModelClass(*args

我们从已有模型中访问参数。当通过Sequential类定义模型时，我们可以通过索引来访问模型的任意层。这 就像模型是一个列表一样，每层的参数都在其属性中。如下所示，我们可以检查第二个全连接层的参数。 print(net[2].state_dict()) OrderedDict([('weight', tensor([[-0.0427, -0.2939, -0.1894, 0.0220, -0.1709, -0.1522, -0 ('2.weight', torch.Size([1, 8])) ('2.bias', �→ torch.Size([1])) 这为我们提供了另一种访问网络参数的方式，如下所示。 net.state_dict()['2.bias'].data tensor([0.0887]) 从嵌套块收集参数 让我们看看，如果我们将多个块相互嵌套，参数命名约定是如何工作的。我们首先定义一个生成块的函数（可 接下来，我们将模型的参数存储在一个叫做“mlp.params”的文件中。 torch.save(net.state_dict(), 'mlp.params') 为了恢复模型，我们实例化了原始多层感知机模型的一个备份。这里我们不需要随机初始化模型参数，而是 直接读取文件中存储的参数。 clone = MLP() clone.load_state_dict(torch.load('mlp.params')) clone

param.detach().cpu().clone() return param # Borrowed from peft.utils.get_peft_model_state_dict def get_peft_state_maybe_zero_3(named_params, bias): if bias == "none": to_return = {k: t for k, t in named_params """Collects the state dict and dump to disk.""" # check if zero3 mode enabled if deepspeed.is_deepspeed_zero3_enabled(): state_dict = trainer.model_wrapped._zero3_consolidated_16bit_state_dict() else: use_lora: (续下页) 1.12. 有监督微调 29 Qwen (接上页) state_dict = get_peft_state_maybe_zero_3( trainer.model.named_parameters(), bias ) else: state_dict = trainer.model.state_dict() if trainer.args.should_save and

查看新表格的后几项 按着 8:2 的比例切分数据集为训练集和测试集： # 切分为训练集和测试集 train_dataset = dataset.sample(frac=0.8,random_state=0) test_dataset = dataset.drop(train_dataset.index) 将 MPG 字段移出为标签数据： # 移动 MPG 油耗效能这一列为真实标签 中实现自定义网络类的计算逻辑即可。自定义网络类继承自 nn.Module 基类，这也是自定义网络类的标准写法，以方便地利用 nn.Module 基类提供的 parameters、load_state_dict、state_dict 等各种便捷功能。网络模型类实现如下： import torch from torch import nn from torch.nn import functional = N_SAMPLES, noise=0.2, random_state=100) # 将 2000 个点按着 7:3 分割为训练集和测试集 X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=TEST_SIZE, random_state=42) print(X.shape, y.shape)

89 5.6 循环层 Recurrent 5.6.1 RNN [source] keras.layers.RNN(cell, return_sequences=False, return_state=False, go_backwards=False, stateful=False, unroll=False) 循环神经网络层基类。 参数 • cell: 一个 RNN 单元实例。RNN 选参数 constants，详见下 面的小节” 关于传递外部常量的注意事项”。 • 一个 state_size 属性。这可以是单个整数（单个状态），在这种情况下，它是循环层状 态的大小（应该与单元输出的大小相同）。这也可以是整数的列表/元组（每个状态一个大 小）。在这种情况下，第一项（state_size [0]）应该与单元输出的大小相同。cell 也可 能是 RNN 单元实例的列表，在这种情况下，RNN 单元实例的列表，在这种情况下，RNN 的单元将堆叠在另一个单元上，实现高 效的堆叠 RNN。 • return_sequences: 布尔值。是返回输出序列中的最后一个输出，还是全部序列。 • return_state: 布尔值。除了输出之外是否返回最后一个状态。 • go_backwards: 布尔值 (默认 False)。如果为 True，则向后处理输入序列并返回相反的序列。 • stateful: 布尔值

chapter is a preview of what to expect in the book. We start off by providing an overview of the state of deep learning, its applications, and rapid growth. We will establish our motivation behind seeking org/challenges/LSVRC/index.php] was an annual competition with the goal of evaluating and discovering state-of-the-art solutions for image classification and object detection, and relied on the ImageNet dataset 04805 (2018). Efficient Deep Learning Deep learning research has been focused on improving on the State of the Art, and as a result we have seen progressive improvements on benchmarks like image classification

Li (SDU) Overview September 6, 2023 35 / 57 Reinforcement Learning (Contd.) Example (Bioreactor) State Current temperature and other sensory readings, composition, target chemical Actions How much heating Overview September 6, 2023 36 / 57 Reinforcement Learning (Contd.) Example (Pick-and-Place Robot) State Current positions and velocities of joints Actions Voltages to apply to motors Reward Reach end-position (SDU) Overview September 6, 2023 37 / 57 Reinforcement Learning (Contd.) Example (Recycling Robot) State charge level of battery Actions Look for cans, wait for can, go recharge Reward Positive for