## Lecture 5: Gaussian Discriminant Analysis, Naive Bayes and EM Algorithm Feng Li Shandong University fli@sdu.edu.cn September 27, 2023 ## Outline  Naive Bayes  Expectation-Maximization distributions • Joint probability distribution • Independence • Conditional probability distribution Bayes' Theorem … ## Sample Space, Events and Probability • A sample space S is the set of all possible

Notes on Gaussian Discriminant Analysis, Naive Bayes and EM Algorithm Feng Li fli@sdu.edu.cn Shandong University, China ## 1 Bayes' Theorem and Inference Bayes’ theorem is stated mathematically as $ p_{X|Y}(x \mid 1) $ according to our assumptions (5)~(7), and make predictions according to Bayes' theorem (see Eq. (2)). Specifically, given a test data featured by $ \tilde{x} $ , we compare always do better than GDA. In practice, logistic regression is used more often than GDA. ## 4 Naive Bayes ### 4.1 Assumption Again, we assume that the m training data are denoted by $ \{x^{(i)}, y^{(i)}\}_{i=1

[-3, -2], [1, 1], [2, 1], [3, 2]]) Y = np.array([1, 1, 1, 2, 2, 2]) # 引入高斯朴素贝叶斯 from sklearn.naive_bayes import GaussianNB # 实例化 clf = GaussianNB() # 训练数据 fit相当于train clf.fit(X, Y) # 输出单个预测结果 print Recognition and Machine Learning[M]. New York: Springer, 2006. [4] Zhang H., The optimality of naïve Bayes[C]//Proceedings of the 17th International Florida Artificial Intelligence Research Society Conference naïve Bayes[C]// Proceedings of the Advances in 14th Neural Information Processing Systems (NIPS), MIT Press, Cambridge, MA, 841-848, 2002. [6] Kohavi R., Scaling up the accuracy of naïve Bayes classifiers:

Greenplum Database MADlib Package 393 Examples 393 Linear Regression 394 Association Rules 395 Naive Bayes Classification 396 References 401 Graph Analytics 401 What is a Graph? 401 Graph Analytics on are examples using the Greenplum MADlib extension: - Linear Regression - Association Rules ## Naive Bayes Classification See the MADlib documentation for additional examples. ## Linear Regression This you might consider placing beer and diapers closer together on the shelves. ## Naive Bayes Classification Naive Bayes analysis predicts the likelihood of an outcome of a class variable, or category,

Greenplum Database MADlib Package 389 Examples 390 Linear Regression 390 Association Rules 391 Naive Bayes Classification 393 References 397 Graph Analytics 397 Graph Analytics 0 What is a Graph? 397 are examples using the Greenplum MADlib extension: - Linear Regression - Association Rules - Naive Bayes Classification See the MADlib documentation for additional examples. ## Linear Regression This you might consider placing beer and diapers closer together on the shelves. ## Naive Bayes Classification Naive Bayes analysis predicts the likelihood of an outcome of a class variable, or category,

Greenplum Database MADlib Package 398 Examples 399 Linear Regression 399 Association Rules 400 Naive Bayes Classification 402 References 406 Graph Analytics 406 Graph Analytics 0 What is a Graph? 406 are examples using the Greenplum MADlib extension: - Linear Regression - Association Rules - Naive Bayes Classification See the MADlib documentation for additional examples. ## Linear Regression This you might consider placing beer and diapers closer together on the shelves. ## Naive Bayes Classification Naive Bayes analysis predicts the likelihood of an outcome of a class variable, or category,

### 1. Scikit-learn概述 ## 监督学习算法-分类 逻辑回归 linear model.LogisticRegression 支持向量机 svm.SVC 朴素贝叶斯 naive bayes.GaussianNB K近邻 neighbors.NearestNeighbors ### 2. Scikit-learn主要用法 ## 监督学习算法-集成学习 sklearn.en

include linear regression, logistic regression, regularization, Gaussian discriminant analysis, Naive Bayes, EM algorithm, SVM, K-means, factor analysis, PCA, neural networks etc. • 68 hours (4 hours/week

逻辑回归（Logistic Regression） • 决策树（Decision Tree） - 随机森林（Random Forest） • 最近邻算法（k-NN） • 朴素贝叶斯（Naive Bayes） - 支持向量机（SVM） • 感知器（Perceptron） • 深度神经网络（DNN） ![Image](/uploads/documents/b/4/3/f/b43f4cf9

probability in the light of the observed data) • Posterior probability of $ \theta $ is given by the Bayes Rule $$ p(\theta\mid D)=\frac{p(\theta)p(D\mid\theta)}{p(D)} $$ • $ p(\theta) $ : Prior probability data (independent of $ \theta $ ) $$ p(D)=\int_{\theta}p(\theta)p(D\mid\theta)d\theta $$ • The Bayes Rule lets us update our belief about $ \theta $ in the light of observed data - While doing MAP