# A novel Information-Driven Strategy for Optimal Regression Assessment

arXiv: https://arxiv.org/abs/2510.14222 · HTML: https://mlsanity.com/paper/2510.14222/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 56 references that are in the map.
- First posted on arXiv 16 Oct 2025.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Sharper bounds for uniformly stable algorithms](https://mlsanity.com/paper/1910.07833/) (2019)
- [Model Complexity of Deep Learning: A Survey](https://mlsanity.com/paper/2103.05127/) (2021)
- [Harder, Better, Faster, Stronger Convergence Rates for Least-Squares Regression](https://mlsanity.com/paper/1602.05419/) (2016)
- [Tree-Values: selective inference for regression trees](https://mlsanity.com/paper/2106.07816/) (2021)
- [Non-attracting Regions of Local Minima in Deep and Wide Neural Networks](https://mlsanity.com/paper/1812.06486/) (2018)
- [Loss Surface Modality of Feed-Forward Neural Network Architectures](https://mlsanity.com/paper/1905.10268/) (2019)
- [On the Use of Cramer-Rao Lower Bound for Least-Variance Circuit Parameters Identification of Li-ion Cells](https://mlsanity.com/paper/2403.10435/) (2024)
- [Fault Detection and Monitoring using a Data-Driven Information-Based Strategy: Method, Theory, and Application](https://mlsanity.com/paper/2405.03667/) (2024)

## Abstract (arXiv)

In Machine Learning (ML), a regression algorithm aims to minimize a loss function based on data. An assessment method in this context seeks to quantify the discrepancy between the optimal response for an input-output system and the estimate produced by a learned predictive model (the student). Evaluating the quality of a learned regressor remains challenging without access to the true data-generating mechanism, as no data-driven assessment method can ensure the achievability of global optimality. This work introduces the Information Teacher, a novel data-driven framework for evaluating regression algorithms with formal performance guarantees to assess global optimality. Our novel approach builds on estimating the Shannon mutual information (MI) between the input variables and the residuals and applies to a broad class of additive noise models. Through numerical experiments, we confirm that the Information Teacher is capable of detecting global optimality, which is aligned with the condition of zero estimation error with respect to the -- inaccessible, in practice -- true model, working as a surrogate measure of the ground truth assessment loss and offering a principled alternative to conventional empirical performance metrics.