Lecture Notes on Support Vector Machine Feng Li fli@sdu.edu.cn Shandong University, China 1 Hyperplane and Margin In a n-dimensional space, a hyper plane is defined by ωT x + b = 0 (1) where ω ∈ Rn Rn is the outward pointing normal vector, and b is the bias term. The n-dimensional space is separated into two half-spaces H+ = {x ∈ Rn | ωT x + b ≥ 0} and H− = {x ∈ Rn | ωT x + b < 0} by the hyperplane margin is defined as γ = min i γ(i) (6) 1 ? ? ! ? ! Figure 1: Margin and hyperplane. 2 Support Vector Machine 2.1 Formulation The hyperplane actually serves as a decision boundary to differentiating

Lecture 6: Support Vector Machine Feng Li Shandong University fli@sdu.edu.cn December 28, 2021 Feng Li (SDU) SVM December 28, 2021 1 / 82 Outline 1 SVM: A Primal Form 2 Convex Optimization Review Hyperplane Separates a n-dimensional space into two half-spaces Defined by an outward pointing normal vector ω ∈ Rn Assumption: The hyperplane passes through origin. If not, have a bias term b; we will then along ω (b < 0 means in opposite direction) Feng Li (SDU) SVM December 28, 2021 3 / 82 Support Vector Machine A hyperplane based linear classifier defined by ω and b Prediction rule: y = sign(ωTx +

takes a 32-bit floating point value in the range [-10.0, 10.0]. We need to transmit a collection (vector) of these variables over an expensive communication channel. Can we use quantization to reduce transmission learnings from the previous exercise into practice. We will code a method `quantize` that quantizes a vector x, given xmin, xmax, and b. It should return the quantized values for a given x. Logistics We just look at how to solve this exercise. We use NumPy for this solution. It supports vector operations which operate on a vector (or a batch) of x variables (vectorized execution) instead of one variable at a

2023 22 / 122 Prediction Based on Bayes’ Theorem X is a random variable indicating the feature vector Y is a random variable indicating the label We perform a trial to obtain a sample x for test, and random An image is represented by a vector of features The feature vectors are random, since the images are randomly given Random variable X representing the feature vector (and thus the image) The labels (deterministic) hypothesis function y = hθ(x) How to model the (probabilistic) relationship between feature vector X and label Y ? P(Y = y | X = x) = P(X = x | Y = y)P(Y = y) P(X = x) Feng Li (SDU) GDA, NB and

below to describe the learning rate, the length of the text, the size of the word vector (each word is translated to a vector) and the locations of initial weights and training checkpoints. A sample text is sentence to a word vector sequence later on. LEARNING_RATE = 0.001 MAX_SEQ_LEN = 500 # The sentences are truncated to this word count. WORD2VEC_LEN = 300 # The size of the word vector CHKPT_DIR = Path('chkpt') represents the number of representative words for a sample text (500 words) and the size of the embedding vector to represent each word (an array of 300 float values) respectively. def create_model(): model =

inputs have similar representations. We will call this representation an Embedding. An embedding is a vector of features that represent aspects of an input numerically. It must fulfill the following goals: such as text, image, audio, video, etc. to a low-dimensional representation such as a fixed length vector of floating point numbers, thus performing dimensionality reduction1. b) The low-dimensional representation two features? In those cases, we could use classical machine learning algorithms like the Support Vector Machine4 (SVM) to learn classifiers that would do this for us. We could rely on deep learning models

a given image. We assume X = [X1, X2, · · · , Xn]T is a random variable representing the feature vector of the given image, and Y ∈ {0, 1} is a random variable representing if there is a cat in the given labeled by y given that the image can be represented by feature vector x, P(X = x | Y = y) is the probability that the image has its feature vector being x given that it is labeled by y, P(Y = y) is the probability logistic regression, we use hypothesis function y = hθ(x) to model the relationship between feature vector x and label y, while we now rely on Byes’ theorem to characterize the relationship through parameters

(GH368) • Can pass extra args, kwds to DataFrame.apply (GH376) • Implement DataFrame.join with vector on argument (GH312) • Added legend boolean flag to DataFrame.plot (GH324) • Can pass multiple levels Select row by location (int) df.ix[loc] Series Slice rows df[5:10] DataFrame Select rows by boolean vector df[bool_vec] DataFrame Row selection, for example, returns a Series whose index is the columns of 332378 DataFrame.sort_index can accept an optional by argument for axis=0 which will use an arbitrary vector or a column name of the DataFrame to determine the sort order: In [131]: df.sort_index(by=’two’)

(GH368) • Can pass extra args, kwds to DataFrame.apply (GH376) • Implement DataFrame.join with vector on argument (GH312) • Added legend boolean flag to DataFrame.plot (GH324) • Can pass multiple levels Select row by location (int) df.ix[loc] Series Slice rows df[5:10] DataFrame Select rows by boolean vector df[bool_vec] DataFrame Row selection, for example, returns a Series whose index is the columns of 157464 DataFrame.sort_index can accept an optional by argument for axis=0 which will use an arbitrary vector or a column name of the DataFrame to determine the sort order: In [131]: df.sort_index(by=’two’)

(GH368) • Can pass extra args, kwds to DataFrame.apply (GH376) • Implement DataFrame.join with vector on argument (GH312) • Added legend boolean flag to DataFrame.plot (GH324) • Can pass multiple levels Select row by location (int) df.ix[loc] Series Slice rows df[5:10] DataFrame Select rows by boolean vector df[bool_vec] DataFrame Row selection, for example, returns a Series whose index is the columns of 151528 DataFrame.sort_index can accept an optional by argument for axis=0 which will use an arbitrary vector or a column name of the DataFrame to determine the sort order: In [131]: df.sort_index(by=’two’)