# On Learning-Curve Monotonicity for Maximum Likelihood Estimators

arXiv: https://arxiv.org/abs/2512.10220 · HTML: https://mlsanity.com/paper/2512.10220/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 40 references that are in the map.
- First posted on arXiv 11 Dec 2025.

## Builds on (most cited first, 13 of 13)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Deep Double Descent: Where Bigger Models and More Data Hurt](https://mlsanity.com/paper/1912.02292/) (2019)
- [Surprises in High-Dimensional Ridgeless Least Squares Interpolation](https://mlsanity.com/paper/1903.08560/) (2019)
- [The generalization error of random features regression: Precise asymptotics and double descent curve](https://mlsanity.com/paper/1908.05355/) (2019)
- [Two models of double descent for weak features](https://mlsanity.com/paper/1903.07571/) (2019)
- [The Shape of Learning Curves: a Review](https://mlsanity.com/paper/2103.10948/) (2021)
- [Martingale Methods for Sequential Estimation of Convex Functionals and Divergences](https://mlsanity.com/paper/2103.09267/) (2021)
- [Minimizers of the Empirical Risk and Risk Monotonicity](https://mlsanity.com/paper/1907.05476/) (2019)
- [Monotone Learning](https://mlsanity.com/paper/2202.05246/) (2022)
- [Are Ensembles Getting Better all the Time?](https://mlsanity.com/paper/2311.17885/) (2023)
- [A universally consistent learning rule with a universally monotone error](https://mlsanity.com/paper/2108.09733/) (2021)

## Built on by (most cited first, 3 of 3)

- [pAI/MSc: ML Theory Research with Humans on the Loop](https://mlsanity.com/paper/2604.20622/) (2026)
- [AI-rithmetic](https://mlsanity.com/paper/2602.10416/) (2026)
- [Sub-Gaussian Concentration and Entropic Normality of the Maximum Likelihood Estimator](https://mlsanity.com/paper/2605.07107/) (2026)

## Abstract (arXiv)

The property of learning-curve monotonicity, highlighted in a recent series of work by Loog, Mey and Viering, describes algorithms which only improve in average performance given more data, for any underlying data distribution within a given family. We establish the first nontrivial monotonicity guarantees for the maximum likelihood estimator in a variety of well-specified parametric settings. For sequential prediction with log loss, we show monotonicity (in fact complete monotonicity) of the forward KL divergence for Gaussian vectors with unknown covariance and either known or unknown mean, as well as for Gamma variables with unknown scale parameter. The Gaussian setting was explicitly highlighted as open in the aforementioned works, even in dimension 1. Finally we observe that for reverse KL divergence, a folklore trick yields monotonicity for very general exponential families. All results in this paper were derived by variants of GPT-5.2 Pro. Humans did not provide any proof strategies or intermediate arguments, but only prompted the model to continue developing additional results, and verified and transcribed its proofs.