Lecture Notes on Gaussian Discriminant Analysis, Naive Bayes and EM Algorithm Feng Li fli@sdu.edu.cn Shandong University, China 1 Bayes’ Theorem and Inference Bayes’ theorem is stated mathematically characterize the relationship through parameters θ = {P(X = x | Y = y), P(Y = y)}x,y. 2 Gaussian Discriminant Analysis In Gaussian Discriminate Analysis (GDA) model, we have the following as- sumptions: compare P(Y = ˜y | X = ˜x) = pY |X(˜y | ˜x) = p(˜x | ˜y)p(˜y) p(˜x) where ˜y = 0, 1. 3 Gaussian Discriminant Analysis and Logistic Regression By far, we introduce two classification algorithms, Logistic

Lecture 5: Gaussian Discriminant Analysis, Naive Bayes and EM Algorithm Feng Li Shandong University fli@sdu.edu.cn September 27, 2023 Feng Li (SDU) GDA, NB and EM September 27, 2023 1 / 122 Outline 5 0 � , µ = � −1 −1.5 � Feng Li (SDU) GDA, NB and EM September 27, 2023 41 / 122 Gaussian Discriminant Analysis (Contd.) Y ∼ Bernoulli(ψ) P(Y = 1) = ψ P(Y = 0) = 1 − ψ Probability mass function = ψy(1 − ψ)1−y, ∀y = 0, 1 Feng Li (SDU) GDA, NB and EM September 27, 2023 42 / 122 Gaussian Discriminant Analysis (Contd.) X | Y = 0 ∼ N(µ0, Σ) Conditional probability density function of X given Y

. • PlayerMove is a type with three variants. In addition to the payloads, Rust will store a discriminant so that it knows at runtime which variant is in a PlayerMove value . • � � � � � � � � � � � � � � � � � � . • Rust � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ) discriminant ( � � � � � � � � � � � � . – � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ) � � � � � � � � � � � � � � � � � � � � � � � � � C) discriminant � � � � � � � � � � � : enum Bar } A, // 0 B = 10000 , C, // 10001 { fn main ) ( } println

Direction::Right. • PlayerMove is a type with three variants. In addition to the payloads, Rust will store a discriminant so that it knows at runtime which variant is in a PlayerMove value. • This might be a good time type as they would if they were all defined in an enum. • Rust uses minimal space to store the discriminant. – If necessary, it stores an integer of the smallest required size – If the allowed variant patterns to encode the discriminant (the ”niche optimization”). For example, Option<&u8> stores either a pointer to an integer or NULL for the None variant. – You can control the discriminant if needed (e.g.

Direction::Right. • PlayerMove is a type with three variants. In addition to the payloads, Rust will store a discriminant so that it knows at runtime which variant is in a PlayerMove value. • This might be a good time type as they would if they were all defined in an enum. • Rust uses minimal space to store the discriminant. – If necessary, it stores an integer of the smallest required size – If the allowed variant patterns to encode the discriminant (the ”niche optimization”). For example, Option<&u8> stores either a pointer to an integer or NULL for the None variant. – You can control the discriminant if needed (e.g.

Clustering (kmeans) – Linear Regression (fitlm) – Logistic & Generalized Linear Regression (fitglm) – Discriminant Analysis Classification (fitcdiscr) – Partition for Cross Validation (cvpartition) – Linear Support

learning. The topics include linear regression, logistic re- gression, regularization, Gaussian discriminant analysis, Naive Bayes, EM algorithm, SVM, K-means, factor analysis, PCA, neural networks etc.

Direction::Right. • PlayerMove is a type with three variants. In addition to the payloads, Rust will store a discriminant so that it knows at runtime which variant is in a PlayerMove value. • This might be a good time 模式对判别标识进行编码（“小众 优化”）。例如，Option<&u8> 存储的要么是指向整数的指针，要么是 None 变体的 NULL 值。 – You can control the discriminant if needed (e.g., for compatibility with C): enum Bar { A, // 0 B = 10000, C, // 10001 } fn println!("B: {}", Bar::B as u32); 50 println!("C: {}", Bar::C as u32); } Without repr, the discriminant type takes 2 bytes, because 10001 fits 2 bytes. 探索更多 Rust 具有多种优化措施，可以减少枚举占用的空间。 • Null pointer

optional as an either 0 or 1 size container of T? std::variant how should we encode the type discriminant? I want to chat with you about some of these things... 🤔2024 Victor Ciura | @ciura_victor

members of the union. In this case, kind was that common property (which is what's considered a discriminant property of Shape ). Checking whether the kind property was "circle" got rid of every type in