distributions Joint probability distribution Independence Conditional probability distribution Bayes’ Theorem ... ... Feng Li (SDU) GDA, NB and EM September 27, 2023 3 / 122 Sample Space, Events and Probability =y(x)dx P(Y = y) = pY (y) Feng Li (SDU) GDA, NB and EM September 27, 2023 16 / 122 Bayes’ Theorem Bayes’ theorem (or Bayes’ rule) describes the probability of an event, based on prior knowledge of conditions A)P(A) P(B) In the Bayesian interpretation, probability measures a “degree of be- lief”, and Bayes’ theorem links the degree of belief in a proposition before and after accounting for evidence. For proposition

thus is a concave function regardless of the original problem; iii) G can be −∞ for some α and β Theorem 1. Lower Bounds Property: If α ⪰ 0, then G(α, β ) ≤ p∗ where p∗ is the optimal value of the (original) We now choose the minimizer of f(˜ω) over all feasible ˜ω’s to get p∗ ≥ G(α, β ). It is shown by Theorem 1 that, the Lagrange dual function provides a non-trivial lower bound to the primal optimization Complementary Slackness Let ω∗ be a primal optimal point and (α∗, β ∗) be a dual optimal point. Theorem 2. Complementary Slackness: If strong duality holds, then α∗ i gi(ω∗) = 0 (16) for ∀i = 1, 2, ·

Bayes and EM Algorithm Feng Li fli@sdu.edu.cn Shandong University, China 1 Bayes’ Theorem and Inference Bayes’ theorem is stated mathematically as the following equation P(A | B) = P(B | A)P(A) P(B) hθ(x) to model the relationship between feature vector x and label y, while we now rely on Byes’ theorem to characterize the relationship through parameters θ = {P(X = x | Y = y), P(Y = y)}x,y. 2 Gaussian and pX|Y (x | 1) according to our assumptions (5)∼(7), and make predictions according to Bayes’ theorem (see Eq. (2)). Specifically, given a test data featured by ˜x, we compare P(Y = ˜y | X = ˜x) = pY

f (x) w.r.t. xi Feng Li (SDU) Linear Regression September 13, 2023 10 / 31 Gradient (Contd.) Theorem For any n-dimensional vector u, the directional derivative of f in the direction of u can be represented BATC Feng Li (SDU) Linear Regression September 13, 2023 26 / 31 Revisiting Least Square (Contd.) Theorem: The matrix ATA is invertible if and only if the columns of A are linearly independent. In this case case, there exists only one least-squares solution θ = (X TX)−1X TY Prove the above theorem in Problem Set 1. Feng Li (SDU) Linear Regression September 13, 2023 27 / 31 Probabilistic Interpretation

"All problems in computer science can be solved by another level of indirection" The fundamental theorem of software engineering (FTSE) 基于 Apache APISIX 破局 What about… Rust + Lua + Nginx? 让 Lua 回归胶水本质！

probability），并用P(B = b | A = a)表示它：它是B = b的概率，前提是A = a已发生。 贝叶斯定理 使用条件概率的定义，我们可以得出统计学中最有用的方程之一：Bayes定理（Bayes’theorem）。根据乘法法 则（multiplication rule ）可得到P(A, B) = P(B | A)P(A)。根据对称性，可得到P(A, B) = P(A | B)P(B)。 假设P(B) 2的可能性将会降低，我们的训练误差将与泛化误差相匹配。 140 4. 多层感知机 统计学习理论 由于泛化是机器学习中的基本问题，许多数学家和理论家毕生致力于研究描述这一现象的形式理论。在同名 定理（eponymous theorem）62中，格里文科和坎特利推导出了训练误差收敛到泛化误差的速率。在一系列 开创性的论文中，Vapnik和Chervonenkis63 将这一理论扩展到更一般种类的函数。这项工作为统计学习理论 验法则相当有用：统计学家认为，能够轻松解释任意事实的模型是复杂的，而表达能力有限但仍能很好地解 62 https://en.wikipedia.org/wiki/Glivenko%E2%80%93Cantelli_theorem 63 https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_theory 4.4. 模型选择、欠拟合和过拟合 141 释数据

协助您决定使用哪一种类型。 7.1.4.1. 设计 设计 Deployment 和 DeploymentConfig 对象之间的一个重要区别是为推出（rollout）过程所选择的 CAP theorem 属性。DeploymentConfig 对象以一致性为先，而 Deployments 对象优先于可用性。 对于 DeploymentConfig 对象，如果运行一个部署器 pod 的节点停机，它不会被替换掉。流程会等待节

协助您决定使用哪一种类型。 8.1.4.1. 设计 设计 Deployment 和 DeploymentConfig 对象之间的一个重要区别是为推出（rollout）过程所选择的 CAP theorem 属性。DeploymentConfig 对象以一致性为先，而 Deployments 对象优先于可用性。 对于 DeploymentConfig 对象，如果运行一个部署器 pod 的节点停机，它不会被替换掉。流程会等待节

file [commands] commands to run If no commands are specified, the client will be put in an interactive mode On Windows: bin\client.bat --help Apache Karaf client -a [port] specify the port to file [commands] commands to run If no commands are specified, the client will be put in an interactive mode Actually, client is a SSH client. You can use any SSH client to connect, like OpenSSH (ssh ntservice.starttype=AUTO_START # Allow the service to interact with the desktop. wrapper.ntservice.interactive=false SystemD The Karaf service wrapper also support Linux SystemD service. 38 INTEGRATION IN

MapReduce Job ... 弱弱的問一下：台灣有多少企業 Cluster 有這麼大？Task 有這麼 多？ 11 / 74 我們對 Hadoop 的期許： Batch Job Interactive Query Real-Time Processing Graph Processing Iterative Modeling 人心不足蛇吞象 Hadoop 的體質 (Batch Processing) Hadoop 從 Batch 應用變成 Data Operating System： 透過 MapReduce 進行 Batch Processing 透過 Hive 與 Tez 進行 Interactive SQL Query ... 15 / 74 MapReduce 改造前 Hadoop 原來的架構，MapReduce 是一切應用的基礎 所有 Job 都得轉換成 MapReduce Initiative 緣由： Hortonworks 主導 希望在 Hadoop 從 MapReduce 演化成 Data Processing Platform 之後，改善 Hive 的效能 滿足 Interactive Query 與 PB-Scale Processing 的需求 三大目標： Speed：比 Hive 10 快 100 倍 Scale：撐的下 TB 到 PB 等級的資料 SQL Compatibility：最廣泛的