Lecture Notes on Support Vector Machine Feng Li fli@sdu.edu.cn Shandong University, China 1 Hyperplane and Margin In a n-dimensional space, a hyper plane is defined by ωT x + b = 0 (1) where ω ∈ Rn the margin is defined as γ = min i γ(i) (6) 1 ? ? ! ? ! Figure 1: Margin and hyperplane. 2 Support Vector Machine 2.1 Formulation The hyperplane actually serves as a decision boundary to differentiating samples are so-called support vector, i.e., the vectors “supporting” the margin boundaries. We can redefine ω by w = � s∈S αsy(s)x(s) where S denotes the set of the indices of the support vectors 4 Kernel

Lecture 6: Support Vector Machine Feng Li Shandong University fli@sdu.edu.cn December 28, 2021 Feng Li (SDU) SVM December 28, 2021 1 / 82 Outline 1 SVM: A Primal Form 2 Convex Optimization Review parallely along ω (b < 0 means in opposite direction) Feng Li (SDU) SVM December 28, 2021 3 / 82 Support Vector Machine A hyperplane based linear classifier defined by ω and b Prediction rule: y = sign(ωTx Scaling ! and " such that min& ' & !() & + " = 1 Feng Li (SDU) SVM December 28, 2021 14 / 82 Support Vector Machine (Primal Form) Maximizing 1/∥ω∥ is equivalent to minimizing ∥ω∥2 = ωTω min ω,b ωTω

→ R 表示二元标量运算符，这意味着该 函数接收两个输入，并产生一个输出。给定同一形状的任意两个向量u和v和二元运算符f，我们可以得到向 量c = F(u, v)。具体计算方法是ci ← f(ui, vi)，其中ci、ui和vi分别是向量c、u和v中的元素。在这里，我们 通过将标量函数升级为按元素向量运算来生成向量值 F : Rd, Rd → Rd。 对于任意具有相同形状的张量，常见的标准算 +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |============================================ 关运算。假设输入的通道数为ci，那么卷积核的输入通道数也需要为ci。如果卷积核的窗口形状是kh × kw，那 么当ci = 1时，我们可以把卷积核看作形状为kh × kw的二维张量。 然而，当ci > 1时，我们卷积核的每个输入通道将包含形状为kh × kw的张量。将这些张量ci连结在一起可以 得到形状为ci × kh × kw的卷积核。由于输入和卷积核都有ci个通道，我们可以对每个通道输入的二维张量和

minimize the within-cluster sum of squares: arg min {Ck} K � i=1 � x∈Ci ∥x − µi∥2 where µi is the mean of points in set Ci How hard is this problem? The problem is NP hard, but there are good heuristic

network layers, deep learning optimizers, data loading utilities, and multi-gpu, and multi-node support. Functions are executed immediately instead of enqueued in a static graph, improving ease of use begin Before you can run an NGC deep learning framework container, your Docker ® environment must support NVIDIA GPUs. To run a container, issue the appropriate command as explained in Running A Container Container and specify the registry, repository, and tags. About this task On a system with GPU support for NGC containers, when you run a container, the following occurs: ‣ The Docker engine loads the image

Experimental support Experimental support Experimental support Supported planned post 2.0 Supported Custom training loop Experimental support Experimental support Support planned post post 2.0 Support planned post 2.0 No support yet Supported Estimator API Limited Support Not supported Limited Support Limited Support Limited Support Limited Support SavedModel：生产级 TensorFlow 模型格式

支持向量机概述 01 支持向量机概述 02 线性可分支持向量机 03 线性支持向量机 04 线性不可分支持向量机 4 1.支持向量机概述 支 持 向 量 机 （ Support Vector Machine, SVM ） 是 一 类 按 监 督 学 习 （ supervised learning）方式对数据进行二元分类的广义线性 分类器（generalized linear 误的情 况。软间隔，就是允许一定量的样本分类错误。 软间隔 硬间隔 线性可分 线性不可分 6 支持向量 1.支持向量机概述 算法思想 找到集合边缘上的若干数据（称为 支持向量（Support Vector）） ，用这些点找出一个平面（称为决 策面），使得支持向量到该平面的 距离最大。 距离 7 1.支持向量机概述 背景知识 任意超平面可以用下面这个线性方程来描述： 大于50000，则使用支 持向量机会非常慢，解决方案是创造、增加更多的特征，然后使用逻辑回归 或不带核函数的支持向量机。 28 参考文献 [1] CORTES C, VAPNIK V. Support-vector networks[J]. Machine learning, 1995, 20(3): 273–297. [2] Andrew Ng. Machine Learning[EB/OL]

state that in this book, we have chosen to work with Tensorflow 2.0 (TF) because it has exhaustive support for building and deploying efficient models on devices ranging from TPUs to edge devices at the time would lead to a 32 / 8 = 4x reduction in space. This fits in well since there is near-universal support for unsigned and signed 8-bit integer data types. 4. The quantized weights are persisted with the addition and subtraction, these gains need to be evaluated in practical settings because they require support from the underlying hardware. Moreover, multiplications and divisions are cheaper at lower precisions

and the TFLite model. Unfortunately we do not (as of the time of writing this chapter) have the support to natively realize the size and latency gains from the clustering. However, we can use gzip to exploit sparse, the matrix multiplication can be performed in half the time. With the advent of hardware support for sparsity and many industrial and academic use cases reporting significant improvements, we feel consecutive values of (the number of bits allocated per value). With that being said, first-class support for clustering in machine learning frameworks like Tensorflow and PyTorch is pending as of the time

later (no GPU support) • Raspbian 9.0 or later �� pip �� TensorFlow tensorflow —Current release for CPU-only (recommended for beginners) tensorflow-gpu —Current release with GPU support (Ubuntu and Windows) Windows) tf-nightly —Nightly build for CPU-only (unstable) tf-nightly-gpu —Nightly build with GPU support (unstable, Ubuntu and Windows) “Hello TensorFlow” Try it “Hello TensorFlow” “Hello TensorFlow”