# A new analytical approach to consistency and overfitting in regularized empirical risk minimization

arXiv: https://arxiv.org/abs/1607.00274 · HTML: https://mlsanity.com/paper/1607.00274/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jul 2016.

## Built on by (most cited first, 4 of 4)

- [The Geometry of Adversarial Training in Binary Classification](https://mlsanity.com/paper/2111.13613/) (2021)
- [On the existence of solutions to adversarial training in multiclass classification](https://mlsanity.com/paper/2305.00075/) (2023)
- [Eikonal depth: an optimal control approach to statistical depths](https://mlsanity.com/paper/2201.05274/) (2022)
- [Higher-Order Regularization Learning on Hypergraphs](https://mlsanity.com/paper/2510.26533/) (2025)

## Abstract (arXiv)

This work considers the problem of binary classification: given training data x1, dots, xn from a certain population, together with associated labels y1,dots, yn ∈ left0,1 right, determine the best label for an element x not among the training data. More specifically, this work considers a variant of the regularized empirical risk functional which is defined intrinsically to the observed data and does not depend on the underlying population. Tools from modern analysis are used to obtain a concise proof of asymptotic consistency as regularization parameters are taken to zero at rates related to the size of the sample. These analytical tools give a new framework for understanding overfitting and underfitting, and rigorously connect the notion of overfitting with a loss of compactness.