(and advantages) • Preview of Numpy & PyTorch & Tensorflow Numpy Tensorflow PyTorch Computation Graph Advantages (continued) • Which one do you think is better? Advantages (continued) • Which one (visualise computation graph) • Various other functions • loss (MSE,CE etc..) • optimizers Prepare Input Data •Load data •Iterate over examples Train Model •Train weights Evaluate Model •Visualise …... Model • In PyTorch, a model is represented by a regular Python class that inherits from the Module class. • Two components • __init__(self): it defines the parts that make up the model —in our

样本分布不均匀 ✗ 隐私保护 • 多个环节 • 多种模型 ✗ 海量参数 ✗ 海量数据 从FM到DeepFM rt 增 加了10倍怎么优化？ 2.模型效果优 化困难 1.方案复杂 Data Model Compute Platform 要求:  准确: 低噪声  全面: 同分布 模型选型:  容量大  计算量小 训练推理:  高qps, 低rt  支持超大模型  性价比 Tensorflow PyTorch Parameter Server MPI TreeModel SQL MapReduce Blink  场景丰富: 图像/视频/推荐/搜索  大数据+大模型: Model Zoo  跨场景+跨模态  开箱即用: 封装复杂性  白盒化, 可扩展性强  积极对接开源系统+模型 FTRL SGD Adam Solutions Librarys 优势: (user/item/attribute)  动态图 标准化: Standard Libraries Graph-Learn: 分布式图算法库 标准化: Standard Solutions Continuous Optimization: Active learning Data Label Model Serving CV / NLP解决方案: EAS Web App Mobile

interesting aspects of the data Examples: Discovering clusters Discovering latent factor Discovering graph structure Matrix completion Feng Li (SDU) Overview September 6, 2023 28 / 57 Unsupervised Learning: Discovering Graph Structures Sometimes we measure a set of correlated variables, and we would like to discover which ones are most correlated with which others This can be represented by a graph, in which (SDU) Overview September 6, 2023 47 / 57 Parametric vs Non-Parametric Models Parametric model We can train a model by using the training data to estimate parameters of it Use these parameters to make predictions

available for Linux from Octave-Forge ). 2 Linear Regression Recall that the linear regression model is hθ(x) = θT x = n � j=0 θjxj, (1) where θ is the parameter which we need to optimize and x is dataset. There are m = 50 training examples, and you will use them to develop a linear regression model using gradient descent algorithm, based on which, we can predict the height given a new age value plotting your results later. We implement linear regression for this problem. The linear regression model in this case is hθ(x) = θT x = 1 � i=0 θixi = θ1x1 + θ2, (4) (1) Implement gradient descent using

learning algorithms help build models, which as the name suggests is an approximate mathematical model of what outputs correspond to a given input. To illustrate, when you visit Netflix’s homepage, the might be popular with other users too. If we train a model to predict the probability based on your behavior and currently trending content, the model will assign a high probability to Seinfeld. While there the performance of the model scaled well with the number of labeled examples, since the network had a large number of parameters. Thus to extract the most out of the setup, the model needed a large number

49 4.3.1 Model 类 API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.3.2 Model 的实用属性 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.3.3 Model 类模型方法 . . . . . . . . . . . . . . . 239 20.8 plot_model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 20.9 multi_gpu_model . . . . . . . . . . . . . . . . . . . . . . Keras 的核心数据结构是 model，一种组织网络层的方式。最简单的模型是 Sequential 顺 序模型，它是由多个网络层线性堆叠的栈。对于更复杂的结构，你应该使用 Keras 函数式 API， 它允许构建任意的神经网络图。 Sequential 顺序模型如下所示： from keras.models import Sequential model = Sequential()

compression techniques. Compression techniques aim to reduce the model footprint (size, latency, memory etc.). We can reduce the model footprint by reducing the number of trainable parameters. However requires many trials and evaluations to reach a smaller model, if it is at all possible. Second, such an approach doesn’t generalize well because the model designs are subjective to the specific problem. In In this chapter, we introduce Quantization, a model compression technique that addresses both these issues. We’ll start with a gentle introduction to the idea of compression. Details of quantization and

multi-gpu, and multi-node support. Functions are executed immediately instead of enqueued in a static graph, improving ease of use and provides a sophisticated debugging experience. In the container, see /workspace/README --shm-size= in the command line to docker run --gpus all To pull data and model descriptions from locations outside the container for use by PyTorch or save results to locations and 2X reduced memory storage for intermediates (reducing the overall memory consumption of your model). Additionally, GEMMs and convolutions with FP16 inputs can run on Tensor Cores, which provide an

非 叶 子 节 点 梯 度 ， 需 使 对 应 张 量 调 用 retain_graph （） 使 用 t e n s o r.grad.zero_() 清 除 张 量 梯 度 如 果 要 保 留 计 算 图 ， 可 通 过 设 置 b a c kw a r d（ ） 中 参 数 retain_graph=True 释 放 计 算 图 具体实例可参考书中2.7小节内容 nn Module Linear Conv* *norm *Aative *Loss functional nn.Sequnce nn.Modelist forward Model() Loss() torch.autograd. backward Torch.optims .step parallel init nn.ModuleDict 定义网络层 构建网络 包还包含了许多别的优化算法。 # Adam构造函数的第一个参数告诉优化器应该更新哪些张量。 learning_rate = 1e-4 optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate) 39 参考文献 1. IAN GOODFELLOW等，《深度学习》，人民邮电出版社，2017 2. Andrew Ng，http://www

参考文献 第 15 章 自定义数据集 15.1 精灵宝可梦数据集 15.2 自定义数据集加载流程 15.3 宝可梦数据集实战 15.4 迁移学习 15.5 Saved_model 15.6 模型部署 15.7 参考文献 预览版202112 人工智能绪论 我们需要的是一台可以从经验中学习的机器。 −阿兰·图灵 1.1 容器可以非常方便地搭建多层的网络。对于 3 层网络，我们可以通过快速 完成 3 层网络的搭建。 # 利用 Sequential 容器封装 3 个网络层，前网络层的输出默认作为下一层的输入 model = nn.Sequential( # 创建第一层，输入为 784，输出为 256 nn.Linear(28*28, 256), nn.ReLU(), # 激活函数 ) 第 1 层的输出节点数设计为 256，第 2 层设计为 128，输出层节点数设计为 10。直接调用 这个模型对象 model(x)就可以返回模型最后一层的输出?。 3.8.2 模型训练 搭建完成 3 层神经网络的对象后，给定输入?，调用 model(?)得到模型输出?后，通过 F.mse_loss 损失函数计算当前的误差ℒ： # 创建优化器，并传递需要优化的参数列表：[w1