Proximal Operator and Optimality Conditions for Ramp Loss SVM

Support vector machines with ramp loss (dubbed as Lr-SVM) have attracted wide attention due to the boundedness of ramp loss. However, the corresponding optimization problem is non-convex and the given Karush-Kuhn-Tucker (KKT) conditions are only the necessary conditions. To enrich the optimality theory of Lr-SVM and go deep into its statistical nature, we first introduce and analyze the proximal operator for ramp loss, and then establish a stronger optimality conditions: P-stationarity, which is proved to be the first-order necessary and sufficient conditions for local minimizer of Lr-SVM. Finally, we define the Lr support vectors based on the concept of P-stationary point, and show that all Lr support vectors fall into the support hyperplanes, which possesses the same feature as the one of hard margin SVM.