Support Vector Machines

This note is about the Support Vector Machines (SVM), a method that has exhibited excellent accuracy in classication problems. Complex mathematics are involved in various aspects of SVM, e.g., proof of its generalization bounds, its optimization, designing and proving the validity of various non-linear kernels, etc. We will, however, not focus on the mathematics. The main purpose of this note is to introduce how the ideas in SVM are formulated, why are they reasonable, and how various simplications are useful in shaping the SVM primal form. We will not touch any generalization bound of SVM, although that is an area which have attracted intensive research eorts. We will not talk about how the optimization problem in SVM can be solved or approximately solved (for eciency and scalability). These choices enable us to focus on the key ideas that lead to SVM, and the strategies in SVM that may be helpful in other domains, and we encourage the readers to also pay attention to these aspects of SVM. primal form. We will not touch any generalization bound of SVM, although that is an area which have attracted intensive research eorts. We will not talk about how the optimization problem in SVM can be solved or approximately solved (for eciency and scalability). These choices enable us to focus on the key ideas that lead to SVM, and the strategies in SVM that may be helpful in other domains, and we encourage the readers to also pay attention to these aspects of SVM.