Lecture Notes on Support Vector Machine Feng Li fli@sdu.edu.cn Shandong University, China 1 Hyperplane and Margin In a n-dimensional space, a hyper plane is defined by ωT x + b = 0 (1) where ω ∈ Rn the margin is defined as γ = min i γ(i) (6) 1 ? ? ! ? ! Figure 1: Margin and hyperplane. 2 Support Vector Machine 2.1 Formulation The hyperplane actually serves as a decision boundary to differentiating samples are so-called support vector, i.e., the vectors “supporting” the margin boundaries. We can redefine ω by w = � s∈S αsy(s)x(s) where S denotes the set of the indices of the support vectors 4 Kernel

Lecture 6: Support Vector Machine Feng Li Shandong University fli@sdu.edu.cn December 28, 2021 Feng Li (SDU) SVM December 28, 2021 1 / 82 Outline 1 SVM: A Primal Form 2 Convex Optimization Review parallely along ω (b < 0 means in opposite direction) Feng Li (SDU) SVM December 28, 2021 3 / 82 Support Vector Machine A hyperplane based linear classifier defined by ω and b Prediction rule: y = sign(ωTx Scaling ! and " such that min& ' & !() & + " = 1 Feng Li (SDU) SVM December 28, 2021 14 / 82 Support Vector Machine (Primal Form) Maximizing 1/∥ω∥ is equivalent to minimizing ∥ω∥2 = ωTω min ω,b ωTω

2011) . . . . . . . . . . . . . . . . . . . . . . . . 122 2 Installation 125 2.1 Python version support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 2.2 Binary data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 4.3 Getting Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 4 Repository: http://github.com/pydata/pandas Issues & Ideas: https://github.com/pydata/pandas/issues Q&A Support: http://stackoverflow.com/questions/tagged/pandas Developer Mailing List: http://groups.google.com/group/pydata

2011) . . . . . . . . . . . . . . . . . . . . . . . . 221 2 Installation 223 2.1 Python version support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 2.2 Installing data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 5.3 Getting Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404 12.7 Unicode Formatting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2011) . . . . . . . . . . . . . . . . . . . . . . . . 95 2 Installation 97 2.1 Python version support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 2.2 Binary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 i 4.3 Getting Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 4 Repository: http://github.com/pydata/pandas Issues & Ideas: https://github.com/pydata/pandas/issues Q&A Support: http://stackoverflow.com/questions/tagged/pandas Developer Mailing List: http://groups.google.com/group/pydata

2011) . . . . . . . . . . . . . . . . . . . . . . . . 63 2 Installation 65 2.1 Python version support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 2.2 Binary data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 4.3 Getting Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 4.4 Repository: http://github.com/pydata/pandas Issues & Ideas: https://github.com/pydata/pandas/issues Q&A Support: http://stackoverflow.com/questions/tagged/pandas Developer Mailing List: http://groups.google.

2011) . . . . . . . . . . . . . . . . . . . . . . . . 166 2 Installation 169 2.1 Python version support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 2.2 Installing data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 4.3 Getting Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 4 Repository: http://github.com/pydata/pandas Issues & Ideas: https://github.com/pydata/pandas/issues Q&A Support: http://stackoverflow.com/questions/tagged/pandas Developer Mailing List: http://groups.google.com/group/pydata

2011) . . . . . . . . . . . . . . . . . . . . . . . . 160 2 Installation 163 2.1 Python version support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 2.2 Installing data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 4.3 Getting Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 4 Repository: http://github.com/pydata/pandas Issues & Ideas: https://github.com/pydata/pandas/issues Q&A Support: http://stackoverflow.com/questions/tagged/pandas Developer Mailing List: http://groups.google.com/group/pydata

Enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.3.1.5 Better support for compressed URLs in read_csv . . . . . . . . . . . . . . . . . 13 1.3.1.6 Pickle file I/O now now supports compression . . . . . . . . . . . . . . . . . . . . . . . 13 1.3.1.7 UInt64 Support Improved . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.3.1.8 GroupBy on Categoricals dtypes will not automatically upcast . . . . . . . . . . . . 27 1.3.2.8 Pandas Google BigQuery support has moved . . . . . . . . . . . . . . . . . . . . . 27 1.3.2.9 Memory Usage for Index is more Accurate

Enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.2.1.5 Better support for compressed URLs in read_csv . . . . . . . . . . . . . . . . . 11 1.2.1.6 Pickle file I/O now now supports compression . . . . . . . . . . . . . . . . . . . . . . . 12 1.2.1.7 UInt64 Support Improved . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.2.1.8 GroupBy on Categoricals dtypes will not automatically upcast . . . . . . . . . . . . 25 1.2.2.8 Pandas Google BigQuery support has moved . . . . . . . . . . . . . . . . . . . . . 26 1.2.2.9 Memory Usage for Index is more Accurate