# Regression with reject option and application to kNN

arXiv: https://arxiv.org/abs/2006.16597 · HTML: https://mlsanity.com/paper/2006.16597/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2020.

## Built on by (most cited first, 30 of 30)

- [Selective Regression Under Fairness Criteria](https://mlsanity.com/paper/2110.15403/) (2021)
- [Regression with Multi-Expert Deferral](https://mlsanity.com/paper/2403.19494/) (2024)
- [Learning to Defer to a Population: A Meta-Learning Approach](https://mlsanity.com/paper/2403.02683/) (2024)
- [Realizable H-Consistent and Bayes-Consistent Loss Functions for Learning to Defer](https://mlsanity.com/paper/2407.13732/) (2024)
- [Selective Prediction via Training Dynamics](https://mlsanity.com/paper/2205.13532/) (2022)
- [Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees for Learning to Defer](https://mlsanity.com/paper/2506.20650/) (2025)
- [Budgeted Multiple-Expert Deferral](https://mlsanity.com/paper/2510.26706/) (2025)
- [Regression with Cost-based Rejection](https://mlsanity.com/paper/2311.04550/) (2023)
- [Survey on Leveraging Uncertainty Estimation Towards Trustworthy Deep Neural Networks: The Case of Reject Option and Post-training Processing](https://mlsanity.com/paper/2304.04906/) (2023)
- [Training Private Models That Know What They Don't Know](https://mlsanity.com/paper/2305.18393/) (2023)
- [Conformalized Selective Regression](https://mlsanity.com/paper/2402.16300/) (2024)
- [Learning to Partially Defer for Sequences](https://mlsanity.com/paper/2502.01459/) (2025)
- [Partial-Label Learning with a Reject Option](https://mlsanity.com/paper/2402.00592/) (2024)
- [Rejection via Learning Density Ratios](https://mlsanity.com/paper/2405.18686/) (2024)
- [When No-Rejection Learning is Consistent for Regression with Rejection](https://mlsanity.com/paper/2307.02932/) (2023)
- [EERO: Early Exit with Reject Option for Efficient Classification with limited budget](https://mlsanity.com/paper/2402.03779/) (2024)
- [From Conformal Predictions to Confidence Regions](https://mlsanity.com/paper/2405.18601/) (2024)
- [Selective Explanations](https://mlsanity.com/paper/2405.19562/) (2024)
- [Beyond the Norms: Detecting Prediction Errors in Regression Models](https://mlsanity.com/paper/2406.06968/) (2024)
- [Optimized Deferral for Imbalanced Settings](https://mlsanity.com/paper/2604.27723/) (2026)
- [Towards Reliable Time Series Forecasting under Future Uncertainty: Ambiguity and Novelty Rejection Mechanisms](https://mlsanity.com/paper/2503.19656/) (2025)
- [A Connection Between Learning to Reject and Bhattacharyya Divergences](https://mlsanity.com/paper/2505.05273/) (2025)
- [Calibrating conditional risk](https://mlsanity.com/paper/2604.20409/) (2026)
- [Robust Adversarial Classification via Abstaining](https://mlsanity.com/paper/2104.02334/) (2021)
- [Variance function estimation in regression model via aggregation procedures](https://mlsanity.com/paper/2110.02715/) (2021)
- [Selective Nonparametric Regression via Testing](https://mlsanity.com/paper/2309.16412/) (2023)
- [Model Agnostic Explainable Selective Regression via Uncertainty Estimation](https://mlsanity.com/paper/2311.09145/) (2023)
- [Bounded-Abstention Pairwise Learning to Rank](https://mlsanity.com/paper/2505.23437/) (2025)
- [Bounded-Abstention Multi-horizon Time-series Forecasting](https://mlsanity.com/paper/2602.04714/) (2026)
- [LOCUS: A Distribution-Free Loss-Quantile Score for Risk-Aware Predictions](https://mlsanity.com/paper/2603.01971/) (2026)

## Abstract (arXiv)

We investigate the problem of regression where one is allowed to abstain from predicting. We refer to this framework as regression with reject option as an extension of classification with reject option. In this context, we focus on the case where the rejection rate is fixed and derive the optimal rule which relies on thresholding the conditional variance function. We provide a semi-supervised estimation procedure of the optimal rule involving two datasets: a first labeled dataset is used to estimate both regression function and conditional variance function while a second unlabeled dataset is exploited to calibrate the desired rejection rate. The resulting predictor with reject option is shown to be almost as good as the optimal predictor with reject option both in terms of risk and rejection rate. We additionally apply our methodology with kNN algorithm and establish rates of convergence for the resulting kNN predictor under mild conditions. Finally, a numerical study is performed to illustrate the benefit of using the proposed procedure.