registry installation documentation based on your platform. ‣ Ensure that you have access and can log in to the NGC container registry. Refer to NGC Getting Started Guide for more information. The deep in.cpp. ‣ ARM ‣ Passing external CUDA Streams to PyTorch via torch.cuda.streams.ExternalStream(stream_v) might fail and is being debugged. PyTorch RN-08516-001_v23.07 | 141 Chapter 22. PyTorch ucx_net_devices. ‣ ARM ‣ Passing external CUDA Streams to PyTorch via `torch.cuda.streams.ExternalStream(stream_v)` might fail and is being debugged. PyTorch RN-08516-001_v23.07 | 148 Chapter 23. PyTorch

unlikely to be the same between two successive pictures. Even though it could be a slight change, it is still a change. The random rotation transformation attempts to simulate that outcome. The random nature baseline500_hist = train(model, tds, vds, epochs=100) Epoch 1/100 2021-11-09 14:44:20.431426: I tensorflow/stream_executor/cuda/cuda_dnn.cc:369] Loaded cuDNN version 8005 32/32 [==============================] baseline1000_hist = train(model, tds, vds, epochs=100) Epoch 1/100 2021-11-09 15:38:34.694059: I tensorflow/stream_executor/cuda/cuda_dnn.cc:369] Loaded cuDNN version 8005 63/63 [==============================]

包含a行和b列的实数矩阵集合 • A ∪ B: 集合A和B的并集 13 • A ∩ B：集合A和B的交集 • A \ B：集合A与集合B相减，B关于A的相对补集 函数和运算符 • f(·)：函数 • log(·)：自然对数 • exp(·): 指数函数 • 1X : 指示函数 • (·)⊤: 向量或矩阵的转置 • X−1: 矩阵的逆 • ⊙: 按元素相乘 • [·, ·]：连结 • |X|：集合的基数 �→'identity_transform', 'independent', 'kl', 'kl_divergence', 'kumaraswamy', 'laplace', 'lkj_cholesky', �→'log_normal', 'logistic_normal', 'lowrank_multivariate_normal', 'mixture_same_family', 'multinomial', 但是我们可以在不改变目标的前提下，通过最大化似然对数来简化。由于历史原因，优化通常是说最小化而 不是最大化。我们可以改为最小化负对数似然− log P(y | X)。由此可以得到的数学公式是： − log P(y | X) = n � i=1 1 2 log(2πσ2) + 1 2σ2 � y(i) − w⊤x(i) − b �2 . (3.1.15) 现在我们只需要假设σ是某个固定常数

= simulate_clustering( x, num_clusters, num_steps=5000, learning_rate=2e-1) The following is the log of the above training. Computing the centroids. Step: 1000, Loss: 0.04999. Step: 2000, Loss: 0.03865 the number of clusters ( ). Figure 5-7 (b) shows the plot. Note that both the x and y axes are in log-scale. Finally, figure 5-7 (c) compares the reconstruction errors between quantization and clustering clustering. The centroids mimic the distribution of x. (b) Change in reconstruction loss as the number of clusters (both x and y axes are in log scale). (c) Comparison of reconstruction errors for both

cluster means µk = mean(Ck) = 1 |Ck| � x∈Ck x Stop when cluster means or the “loss” does not change by much Feng Li (SDU) K-Means December 28, 2021 8 / 46 K-means: Initialization (assume K = 2) Can also information criterion such as AIC (Akaike Information Crite- rion) AIC = 2L(ˆµ, X, ˆZ) + K log D and choose the K that has the smallest AIC (discourages large K) Feng Li (SDU) K-Means December

Assistant Response 1:') responses = [] for responses in llm.chat(messages=messages, functions=functions, stream=True): print(responses) messages.extend(responses) # extend conversation with assistant's reply llm.chat( (续下页) 1.13. Function Calling 39 Qwen (接上页) messages=messages, functions=functions, stream=True, ): # get a new response from the model where it can see the function response print(responses)

122 Warm Up (Contd.) Log-likelihood function ℓ(θ) = log m � i=1 pX,Y (x(i), y(i)) = log m � i=1 pX|Y (x(i) | y(i))pY (y(i)) = m � i=1 � log pX|Y (x(i) | y(i)) + log pY (y(i)) � where θ = {pX|Y Given m sample data, the log-likelihood is ℓ(ψ, µ0, µ1, Σ) = log m � i=1 pX,Y (x(i), y(i); ψ, µ0, µ1, Σ) = log m � i=1 pX|Y (x(i) | y(i); µ0, µ1, Σ)pY (y(i); ψ) = m � i=1 log pX|Y (x(i) | y(i); µ0 + m � i=1 log pY (y(i); ψ) Feng Li (SDU) GDA, NB and EM September 27, 2023 46 / 122 Gaussian Discriminant Analysis (Contd.) The log-likelihood function ℓ(ψ, µ0, µ1, Σ) = m � i=1 log pX|Y (x(i) |

y(i))}i=1,··· ,m, the log-likelihood is defined as ℓ(ψ, µ0, µ1, Σ) = log m � i=1 pX,Y (x(i), y(i); ψ, µ0, µ1, Σ) = log m � i=1 pX|Y (x(i) | y(i); µ0, µ1, Σ)pY (y(i); ψ) = m � i=1 log pX|Y (x(i) | y(i); + m � i=1 log pY (y(i); ψ)(8) where ψ, µ0, and σ are parameters. Substituting Eq. (5)∼(7) into Eq. (8) gives 2 us a full expression of ℓ(ψ, µ0, µ1, Σ) ℓ(ψ, µ0, µ1, Σ) = m � i=1 log pX|Y (x(i) | m � i=1 log pY (y(i); ψ) = � i:y(i)=0 log � 1 (2π)n/2|Σ|1/2 exp � −1 2(x − µ0)T Σ−1(x − µ0) �� + � i:y(i)=1 log � 1 (2π)n/2|Σ|1/2 exp � −1 2(x − µ1)T Σ−1(x − µ1) �� + m � i=1 log ψy(i)(1 −

J(θ) = − 1 m m � i=1 [y(i) log(hθ(x(i))) + (1 − y(i)) log(1 − hθ(x(i)))] Adding a term for regularization J(θ) = − 1 m m � i=1 [y(i) log(hθ(x(i)))+(1−y(i)) log(1−hθ(x(i)))]+ λ 2m n � j=1 θ2 m � i=1 p(d(i); θ) MLE typically maximizes the log-likelihood instead of the likelihood ℓ(θ) = log L(θ) = log m � i=1 p(d(i); θ) = m � i=1 log p(d(i); θ) Maximum likelihood parameter estimation estimation θMLE = arg max θ ℓ(θ) = arg max θ m � i=1 log p(d(i); θ) Feng Li (SDU) Regularization and Bayesian Statistics September 20, 2023 13 / 25 Maximum-a-Posteriori Estimation (MAP) Maximum-a-Posteriori

对数运算，例如常见的loge 、log2 、log1 等，可以直接调用 torch.log()、 torch.log2()、torch.log10()等函数实现。自然对数loge 实现如下: In [98]: x = torch.arange(3).float() # 转换为浮点数 x = torch.exp(x) # 先指数运算 torch.log(x) # 再对数运算 Out[98]: Out[98]: tensor([0., 1., 2.]) 如果希望计算其它底数的对数，可以根据对数的换底公式： log? = loge loge ? 间接地通过 torch.log()实现。这里假设不调用 torch.log10()函数，通过换底公式算 loge ? loge 1 来间 接计，实现如下： In [99]: x = torch.tensor([1 PyTorch 基础 36 x = 10**x # 指数运算 torch.log(x)/torch.log(torch.tensor(10.)) # 换底公式计算 Log10 Out[99]: tensor([1., 2.]) 实现起来并不麻烦。实际中通常使用 torch.log()函数就够了。 4.9.4 矩阵相乘运算 神经网络中间包含了大量的矩阵相乘