# Controlled abstention neural networks for identifying skillful predictions for regression problems

arXiv: https://arxiv.org/abs/2104.08236 · HTML: https://mlsanity.com/paper/2104.08236/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 44 references that are in the map.
- First posted on arXiv 16 Apr 2021.

## Builds on (most cited first, 8 of 8)

- [SelectiveNet: A Deep Neural Network with an Integrated Reject Option](https://mlsanity.com/paper/1901.09192/) (2019)
- [Deep learning to represent sub-grid processes in climate models](https://mlsanity.com/paper/1806.04731/) (2018)
- [Combating Label Noise in Deep Learning Using Abstention](https://mlsanity.com/paper/1905.10964/) (2019)
- [Stochastic Super-Resolution for Downscaling Time-Evolving Atmospheric Fields with a Generative Adversarial Network](https://mlsanity.com/paper/2005.10374/) (2020)
- [Machine Learning for Stochastic Parameterization: Generative Adversarial Networks in the Lorenz '96 Model](https://mlsanity.com/paper/1909.04711/) (2019)
- [Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset](https://mlsanity.com/paper/2103.10005/) (2021)
- [Resampling with neural networks for stochastic parameterization in multiscale systems](https://mlsanity.com/paper/2004.01457/) (2020)
- [A Unified Plug-and-Play Framework for Effective Data Denoising and Robust Abstention](https://mlsanity.com/paper/2009.12027/) (2020)

## Built on by (most cited first, 9 of 9)

- [Evidential Deep Learning: Enhancing Predictive Uncertainty Estimation for Earth System Science Applications](https://mlsanity.com/paper/2309.13207/) (2023)
- [CIRA Guide to Custom Loss Functions for Neural Networks in Environmental Sciences -- Version 1](https://mlsanity.com/paper/2106.09757/) (2021)
- [Evaluation of Machine Learning Techniques for Forecast Uncertainty Quantification](https://mlsanity.com/paper/2111.14844/) (2021)
- [Survey on Leveraging Uncertainty Estimation Towards Trustworthy Deep Neural Networks: The Case of Reject Option and Post-training Processing](https://mlsanity.com/paper/2304.04906/) (2023)
- [Uncertainty Quantification of Wind Gust Predictions in the Northeast United States: An Evidential Neural Network and Explainable Artificial Intelligence Approach](https://mlsanity.com/paper/2502.00300/) (2025)
- [Handling Label Uncertainty on the Example of Automatic Detection of Shepherd's Crook RCA in Coronary CT Angiography](https://mlsanity.com/paper/2306.01752/) (2023)
- [Abstaining Machine Learning -- Philosophical Considerations](https://mlsanity.com/paper/2409.00706/) (2024)
- [Predicting Tropical Cyclone Track Forecast Errors using a Probabilistic Neural Network](https://mlsanity.com/paper/2503.09840/) (2025)
- [Bayesian Deep Learning for Convective Initiation Nowcasting Uncertainty Estimation](https://mlsanity.com/paper/2507.16219/) (2025)

## Abstract (arXiv)

The earth system is exceedingly complex and often chaotic in nature, making prediction incredibly challenging: we cannot expect to make perfect predictions all of the time. Instead, we look for specific states of the system that lead to more predictable behavior than others, often termed "forecasts of opportunity". When these opportunities are not present, scientists need prediction systems that are capable of saying "I don't know." We introduce a novel loss function, termed "abstention loss", that allows neural networks to identify forecasts of opportunity for regression problems. The abstention loss works by incorporating uncertainty in the network's prediction to identify the more confident samples and abstain (say "I don't know") on the less confident samples. The abstention loss is designed to determine the optimal abstention fraction, or abstain on a user-defined fraction via a PID controller. Unlike many methods for attaching uncertainty to neural network predictions post-training, the abstention loss is applied during training to preferentially learn from the more confident samples. The abstention loss is built upon a standard computer science method. While the standard approach is itself a simple yet powerful tool for incorporating uncertainty in regression problems, we demonstrate that the abstention loss outperforms this more standard method for the synthetic climate use cases explored here. The implementation of proposed loss function is straightforward in most network architectures designed for regression, as it only requires modification of the output layer and loss function.