Lecture Notes on Gaussian Discriminant Analysis, Naive Bayes and EM Algorithm Feng Li fli@sdu.edu.cn Shandong University, China 1 Bayes’ Theorem and Inference Bayes’ theorem is stated mathematically through parameters θ = {P(X = x | Y = y), P(Y = y)}x,y. 2 Gaussian Discriminant Analysis In Gaussian Discriminate Analysis (GDA) model, we have the following as- sumptions: • A1: Y ∼ Bernoulli(ψ): Y follows ˜y | X = ˜x) = pY |X(˜y | ˜x) = p(˜x | ˜y)p(˜y) p(˜x) where ˜y = 0, 1. 3 Gaussian Discriminant Analysis and Logistic Regression By far, we introduce two classification algorithms, Logistic Regression

Lecture 5: Gaussian Discriminant Analysis, Naive Bayes and EM Algorithm Feng Li Shandong University fli@sdu.edu.cn September 27, 2023 Feng Li (SDU) GDA, NB and EM September 27, 2023 1 / 122 Outline Outline 1 Probability Theory Review 2 A Warm-Up Case 3 Gaussian Discriminate Analysis 4 Naive Bayes 5 Expectation-Maximization (EM) Algorithm Feng Li (SDU) GDA, NB and EM September 27, 2023 2 / 122 = � −1 −1.5 � Feng Li (SDU) GDA, NB and EM September 27, 2023 41 / 122 Gaussian Discriminant Analysis (Contd.) Y ∼ Bernoulli(ψ) P(Y = 1) = ψ P(Y = 0) = 1 − ψ Probability mass function pY (y) = ψy(1

These techniques help to overcome dataset shortcomings like: small size, skewed samples, or partial coverage. It is fair to ask: why don’t we just get more data? Consider the following examples. MNIST dataset sentiments of the original and the transformed sentences are consistent. This can be used for sentiment analysis. This transformation has two main implications. First, it augments our dataset with additional examples that the agreement between the text and the original label is intact. In the context of sentiment analysis, the transformation must preserve the original sentiment of the text. For a language translation

skip connections Decoder Details from multi-frames Analysis 52 3 identical frames Output (identical) Details from multi-frames Analysis 53 3 consecutive frames Output (consecutive) Output Layer v.s. Baseline Analysis 54 Output (baseline) ????????????????????????→0 BW Resize Backward warping + Resize (baseline) Ablation Study: SPMC Layer v.s. Baseline Analysis 55 Output (SPMC) deconv 86 Data from GOPRO dataset Using Different Number of Scales Analysis 87 1 scale Input 2 scales 3 scales Baseline Models Analysis 88 Model SS SC w/o R RNN SR-Flat Param 2.73M 8.19M 2.73M 3.03M

which makes deep learning a very powerful tool. Deep learning algorithms are also useful for the analysis of unstructured data. Let us go through the basics of deep learning in this chapter. Artificial floatx() 'float32' Let us understand some of the significant backend functions used for data analysis in brief: get_uid() It is the identifier for the default graph. It is defined below: >>> It is used to convert positive into dense vectors of fixed size. Its main application is in text analysis. The signature of the Embedding layer function and its arguments with default value is as follows

regression, logistic re- gression, regularization, Gaussian discriminant analysis, Naive Bayes, EM algorithm, SVM, K-means, factor analysis, PCA, neural networks etc. 68 hours (4 hours/week × 17 weeks) Labs Personalized tutoring Discover new knowledge from large databases (data mining) Market basket analysis (e.g. diapers and beer) Medical information mining (e.g. migraines to calcium channel blockers to these with fewer ones, without loss of information. On simple way is to use PCA (Principal Component Analysis) Suppose that all data are in a space, we first find the direction of high- est variance of these

我们 能把它们分成风景照片、狗、婴儿、猫和山峰的照片吗？同样，给定一组用户的网页浏览记录，我们能 否将具有相似行为的用户聚类呢？ • 主成分分析（principal component analysis）问题：我们能否找到少量的参数来准确地捕捉数据的线 性相关属性？比如，一个球的运动轨迹可以用球的速度、直径和质量来描述。再比如，裁缝们已经开发 出了一小部分参数，这些参数相当准确地描述了人 1中选择哪一个呢？事实上，他们都一样的好。然而，如果我们只选择一个偏移量， 那么用于训练网络的、所有可能的子序列的覆盖范围将是有限的。因此，我们可以从随机偏移量开始划分序 列，以同时获得覆盖性（coverage）和随机性（randomness）。下面，我们将描述如何实现随机采样（random sampling）和 顺序分区（sequential partitioning）策略。 308 8. 循环神经网络 神经网络和卷积神经网络的模型设计。 15.1 情感分析及数据集 随着在线社交媒体和评论平台的快速发展，大量评论的数据被记录下来。这些数据具有支持决策过程的巨大 潜力。情感分析（sentiment analysis）研究人们在文本中（如产品评论、博客评论和论坛讨论等）“隐藏”的 情绪。它在广泛应用于政治（如公众对政策的情绪分析）、金融（如市场情绪分析）和营销（如产品研究和品 牌管理）等领域。 由于

estate purchase predictions FINRA • Anomaly detection, sequence matching, regression analysis, network/tribe analysis Netflix • Recommendation engine Pinterest • Image recognition search Fraud.net •

文本方面的词袋模型、词嵌入模型等 3. 特征提取 18 许永洪,吴林颖.中国各地区人口特征和房价波动的动态关系[J].统计研究,2019,36(01) 1.PCA(Principal Component Analysis，主成分分析) PCA 是降维最经典的方法，它旨在是找到数据中的主成分，并利 用这些主成分来表征原始数据，从而达到降维的目的。 PCA 的思想是通过坐标轴转换，寻找数据分布的最优子空间。 3. 特征提取 降维 19 许永洪,吴林颖.中国各地区人口特征和房价波动的动态关系[J].统计研究,2019,36(01) 2. ICA(Independent Component Analysis，独立成分分析) ICA独立成分分析，获得的是相互独立的属性。ICA算法本质寻找一 个线性变换 ? = ??，使得 ? 的各个特征分量之间的独立性最大。 PCA 对数据 进行降维 ICA

PCA(主成分分析) 01 降维概述 02 SVD(奇异值分解) 03 PCA(主成分分析) 31 3.PCA(主成分分析) 主成分分析（Principal Component Analysis,PCA）是一种降维方法， 通过将一个大的特征集转换成一个较小的特征集，这个特征集仍然包含 了原始数据中的大部分信息，从而降低了原始数据的维数。 减少一个数据集的特征数量自然是以牺牲准确性为代价的，但降维的诀 Dimensionality of Data with Neural Networks.[J]. Science, 2006. [3] Jolliffe I T . Principal Component Analysis[J]. Journal of Marketing Research, 2002, 87(4):513. [4] 李航. 统计学习方法[M]. 北京: 清华大学出版社,2019. [5]