# Learning with Smooth Hinge Losses

arXiv: https://arxiv.org/abs/2103.00233 · HTML: https://mlsanity.com/paper/2103.00233/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 47 references that are in the map.
- First posted on arXiv 27 Feb 2021.

## Builds on (most cited first, 7 of 7)

- [SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives](https://mlsanity.com/paper/1407.0202/) (2014)
- [Minimizing Finite Sums with the Stochastic Average Gradient](https://mlsanity.com/paper/1309.2388/) (2013)
- [Accelerated Proximal Stochastic Dual Coordinate Ascent for Regularized Loss Minimization](https://mlsanity.com/paper/1309.2375/) (2013)
- [Distributed Inference for Linear Support Vector Machine](https://mlsanity.com/paper/1811.11922/) (2018)
- [Finito: A Faster, Permutable Incremental Gradient Method for Big Data Problems](https://mlsanity.com/paper/1407.2710/) (2014)
- [More Efficient Estimation for Logistic Regression with Optimal Subsample](https://mlsanity.com/paper/1802.02698/) (2018)
- [Scaling Up Sparse Support Vector Machines by Simultaneous Feature and Sample Reduction](https://mlsanity.com/paper/1607.06996/) (2016)

## Built on by (most cited first, 9 of 9)

- [Synthetic Face Datasets Generation via Latent Space Exploration from Brownian Identity Diffusion](https://mlsanity.com/paper/2405.00228/) (2024)
- [Harmonic Loss Trains Interpretable AI Models](https://mlsanity.com/paper/2502.01628/) (2025)
- [Nonparametric estimation of Hawkes processes with RKHSs](https://mlsanity.com/paper/2411.00621/) (2024)
- [Density-Aware Personalized Training for Risk Prediction in Imbalanced Medical Data](https://mlsanity.com/paper/2207.11382/) (2022)
- [Xtreme Margin: A Tunable Loss Function for Binary Classification Problems](https://mlsanity.com/paper/2211.00176/) (2022)
- [Coding for Intelligence from the Perspective of Category](https://mlsanity.com/paper/2407.01017/) (2024)
- [Online AUC Optimization Based on Second-order Surrogate Loss](https://mlsanity.com/paper/2510.21202/) (2025)
- [Enhancing Cross Domain SAR Oil Spill Segmentation via Morphological Region Perturbation and Synthetic Label-to-SAR Generation](https://mlsanity.com/paper/2512.02290/) (2025)
- [The geometry of invariant learning: an information-theoretic analysis of data augmentation and generalization](https://mlsanity.com/paper/2602.14423/) (2026)

## Abstract (arXiv)

Due to the non-smoothness of the Hinge loss in SVM, it is difficult to obtain a faster convergence rate with modern optimization algorithms. In this paper, we introduce two smooth Hinge losses psiG(α;σ) and psiM(α;σ) which are infinitely differentiable and converge to the Hinge loss uniformly in α as σ tends to 0. By replacing the Hinge loss with these two smooth Hinge losses, we obtain two smooth support vector machines(SSVMs), respectively. Solving the SSVMs with the Trust Region Newton method (TRON) leads to two quadratically convergent algorithms. Experiments in text classification tasks show that the proposed SSVMs are effective in real-world applications. We also introduce a general smooth convex loss function to unify several commonly-used convex loss functions in machine learning. The general framework provides smooth approximation functions to non-smooth convex loss functions, which can be used to obtain smooth models that can be solved with faster convergent optimization algorithms.