# Local Duality for Sparse Support Vector Machines

arXiv: https://arxiv.org/abs/2601.20170 · HTML: https://mlsanity.com/paper/2601.20170/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 52 references that are in the map.
- First posted on arXiv 28 Jan 2026.

## Builds on (most cited first, 9 of 9)

- [Learning Using Privileged Information: SVM+ and Weighted SVM](https://mlsanity.com/paper/1306.3161/) (2013)
- [Cardinality Minimization, Constraints, and Regularization: A Survey](https://mlsanity.com/paper/2106.09606/) (2021)
- [Global and Quadratic Convergence of Newton Hard-Thresholding Pursuit](https://mlsanity.com/paper/1901.02763/) (2019)
- [Multiclass Optimal Classification Trees with SVM-splits](https://mlsanity.com/paper/2111.08674/) (2021)
- [Quadratic Convergence of Smoothing Newton's Method for 0/1 Loss Optimization](https://mlsanity.com/paper/2103.14987/) (2021)
- [Mixed-Integer Quadratic Optimization and Iterative Clustering Techniques for Semi-Supervised Support Vector Machines](https://mlsanity.com/paper/2303.12532/) (2023)
- [Algorithms for Sparse Support Vector Machines](https://mlsanity.com/paper/2110.07691/) (2021)
- [Sparse SVM with Hard-Margin Loss: a Newton-Augmented Lagrangian Method in Reduced Dimensions](https://mlsanity.com/paper/2307.16281/) (2023)
- [Composite Optimization with Indicator Functions: Stationary Duality and a Semismooth Newton Method](https://mlsanity.com/paper/2506.08374/) (2025)

## Abstract (arXiv)

Due to the rise of cardinality minimization in optimization, sparse support vector machines (SSVMs) have attracted much attention lately and show certain empirical advantages over convex SVMs. A common way to derive an SSVM is to add a cardinality function such as ℓ0-norm to the dual problem of a convex SVM. However, this process lacks theoretical justification. This paper fills the gap by developing a local duality theory for such an SSVM formulation and exploring its relationship with the hinge-loss SVM (hSVM) and the ramp-loss SVM (rSVM). In particular, we prove that the derived SSVM is exactly the dual problem of the 0/1-loss SVM, and the linear representer theorem holds for their local solutions. The local solution of SSVM also provides guidelines on selecting hyperparameters of hSVM and rSVM. Under specific conditions, we show that a sequence of global solutions of hSVM converges to a local solution of 0/1-loss SVM. Moreover, a local minimizer of 0/1-loss SVM is a local minimizer of rSVM. This explains why a local solution induced by SSVM outperforms hSVM and rSVM in the prior empirical study. We further conduct numerical tests on real datasets and demonstrate potential advantages of SSVM by working with locally nice solutions proposed in this paper.