Y. Lin (1999)
Support Vector Machines and the Bayes Rule in Classification
University of Wisconsin-Madison, Department of statistics technical report(1014).
The Bayes rule is the optimal classification rule if the underlying distribution of the data is known. In practice we do not know the underlying distribution, and need to ``learn'' classification rules from the data. One way to derive classification rules in practice is to implement the Bayes rule approximately by estimating an appropriate classification function. Traditional statistical methods use estimated log odds ratio as the classification function. Support vector machines (SVMs) are one type of large margin classifier, and the relationship between SVMs and the Bayes rule was not clear. In this paper, it is shown that SVMs implement the Bayes rule approximately by targeting at some interesting classification functions. This helps understand the success of SVMs in many classification studies, and makes it easier to compare SVMs and traditional statistical methods.