# No Unique Minimizer, No Problem: On the Consistency of Robust Neural Classifiers

arXiv: https://arxiv.org/abs/2608.08489 · HTML: https://mlsanity.com/paper/2608.08489/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 9 Aug 2026.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [The Loss Surfaces of Multilayer Networks](https://mlsanity.com/paper/1412.0233/) (2014)
- [AlphaNet: Improved Training of Supernets with Alpha-Divergence](https://mlsanity.com/paper/2102.07954/) (2021)
- [Limits of Convergence-Rate Control for Open-Weight Safety](https://mlsanity.com/paper/2602.18868/) (2026)
- [The Minimum S-Divergence Estimator under Continuous Models: The Basu-Lindsay Approach](https://mlsanity.com/paper/1408.1239/) (2014)
- [Joint Estimation and Inference for Data Integration Problems based on Multiple Multi-layered Gaussian Graphical Models](https://mlsanity.com/paper/1803.03348/) (2018)
- [Introducing Fractional Classification Loss for Robust Learning with Noisy Labels](https://mlsanity.com/paper/2508.06346/) (2025)
- [Provably robust learning of regression neural networks using β-divergences](https://mlsanity.com/paper/2602.08933/) (2026)
- [Penalized Maximum Likelihood Estimation of Multi-layered Gaussian Graphical Models](https://mlsanity.com/paper/1601.00736/) (2016)
- [The alpha-beta divergence for real and complex data](https://mlsanity.com/paper/2508.03272/) (2025)
- [rSDNet: Unified Robust Neural Learning against Label Noise and Adversarial Attacks](https://mlsanity.com/paper/2603.17628/) (2026)

## Abstract (arXiv)

Neural network classifiers trained by cross-entropy minimization are highly sensitive to label noise and adversarial contamination. While robust alternatives offer bounded influence and resistance to corruption, their statistical foundations in the deep learning setting are insufficient due to a fundamental difficulty: neural parameterizations are non-identifiable, so the population loss minimizer is an equivalence class of parameters, not a unique point. We develop a consistency theory for robust neural classifiers based on the S-divergence family that requires no identifiability assumption. Casting training as stochastic optimization over a non-identifiable parameter space, we prove that empirical S-divergence minimizers converge to the population-optimal equivalence class under mild regularity conditions, and verify these conditions for three architecture choices. We further establish that limit points of the robust training algorithm are stationary points of the empirical objective. Experiments on vision and language benchmark datasets confirm that S-divergence training maintains clean-data accuracy while exhibiting performance competitive with existing robust methods.