# Learning with Symmetric Label Noise: The Importance of Being Unhinged

arXiv: https://arxiv.org/abs/1505.07634 · HTML: https://mlsanity.com/paper/1505.07634/

## Key facts

- Cited by 120 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2015.

## Built on by (most cited first, 30 of 120)

- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Decoupled Knowledge Distillation](https://mlsanity.com/paper/2203.08679/) (2022)
- [ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks](https://mlsanity.com/paper/2102.11600/) (2021)
- [Learning with Feature-Dependent Label Noise: A Progressive Approach](https://mlsanity.com/paper/2103.07756/) (2021)
- [Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark](https://mlsanity.com/paper/2311.06750/) (2023)
- [Fisher SAM: Information Geometry and Sharpness Aware Minimisation](https://mlsanity.com/paper/2206.04920/) (2022)
- [FINE Samples for Learning with Noisy Labels](https://mlsanity.com/paper/2102.11628/) (2021)
- [Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization](https://mlsanity.com/paper/2102.02414/) (2021)
- [Open-set Label Noise Can Improve Robustness Against Inherent Label Noise](https://mlsanity.com/paper/2106.10891/) (2021)
- [DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations](https://mlsanity.com/paper/2201.09637/) (2022)
- [Towards Label-free Scene Understanding by Vision Foundation Models](https://mlsanity.com/paper/2306.03899/) (2023)
- [Asymmetric Loss Functions for Learning with Noisy Labels](https://mlsanity.com/paper/2106.03110/) (2021)
- [Investigating Why Contrastive Learning Benefits Robustness Against Label Noise](https://mlsanity.com/paper/2201.12498/) (2022)
- [Contrastive Learning Improves Model Robustness Under Label Noise](https://mlsanity.com/paper/2104.08984/) (2021)
- [Unsupervised Domain Adaptation of Black-Box Source Models](https://mlsanity.com/paper/2101.02839/) (2021)
- [Meta-learning with an Adaptive Task Scheduler](https://mlsanity.com/paper/2110.14057/) (2021)
- [Robust Imitation Learning from Noisy Demonstrations](https://mlsanity.com/paper/2010.10181/) (2020)
- [Towards Federated Learning against Noisy Labels via Local Self-Regularization](https://mlsanity.com/paper/2208.12807/) (2022)
- [Noise-resistant Deep Metric Learning with Ranking-based Instance Selection](https://mlsanity.com/paper/2103.16047/) (2021)
- [Few-shot Learning with Noisy Labels](https://mlsanity.com/paper/2204.05494/) (2022)
- [Noise-robust Graph Learning by Estimating and Leveraging Pairwise Interactions](https://mlsanity.com/paper/2106.07451/) (2021)
- [Is BERT Robust to Label Noise? A Study on Learning with Noisy Labels in Text Classification](https://mlsanity.com/paper/2204.09371/) (2022)
- [Early Stopping Against Label Noise Without Validation Data](https://mlsanity.com/paper/2502.07551/) (2025)
- [Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection](https://mlsanity.com/paper/2111.14932/) (2021)
- [Understanding the Interaction of Adversarial Training with Noisy Labels](https://mlsanity.com/paper/2102.03482/) (2021)
- [Informative Pseudo-Labeling for Graph Neural Networks with Few Labels](https://mlsanity.com/paper/2201.07951/) (2022)
- [Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence](https://mlsanity.com/paper/2105.01798/) (2021)
- [Geometric-Averaged Preference Optimization for Soft Preference Labels](https://mlsanity.com/paper/2409.06691/) (2024)
- [Late Stopping: Avoiding Confidently Learning from Mislabeled Examples](https://mlsanity.com/paper/2308.13862/) (2023)
- [Delving into Sample Loss Curve to Embrace Noisy and Imbalanced Data](https://mlsanity.com/paper/2201.00849/) (2021)

## Abstract (arXiv)

Convex potential minimisation is the de facto approach to binary classification. However, Long and Servedio [2010] proved that under symmetric label noise (SLN), minimisation of any convex potential over a linear function class can result in classification performance equivalent to random guessing. This ostensibly shows that convex losses are not SLN-robust. In this paper, we propose a convex, classification-calibrated loss and prove that it is SLN-robust. The loss avoids the Long and Servedio [2010] result by virtue of being negatively unbounded. The loss is a modification of the hinge loss, where one does not clamp at zero; hence, we call it the unhinged loss. We show that the optimal unhinged solution is equivalent to that of a strongly regularised SVM, and is the limiting solution for any convex potential; this implies that strong l2 regularisation makes most standard learners SLN-robust. Experiments confirm the SLN-robustness of the unhinged loss.