# Filtering instances and rejecting predictions to obtain reliable models in healthcare

arXiv: https://arxiv.org/abs/2510.24368 · HTML: https://mlsanity.com/paper/2510.24368/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 62 references that are in the map.
- First posted on arXiv 28 Oct 2025.

## Builds on (most cited first, 18 of 18)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Deep Learning on a Data Diet: Finding Important Examples Early in Training](https://mlsanity.com/paper/2107.07075/) (2021)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [When Do Neural Nets Outperform Boosted Trees on Tabular Data?](https://mlsanity.com/paper/2305.02997/) (2023)
- [Data-centric Artificial Intelligence: A Survey](https://mlsanity.com/paper/2303.10158/) (2023)
- [Machine Learning with a Reject Option: A survey](https://mlsanity.com/paper/2107.11277/) (2021)
- [Optimal strategies for reject option classifiers](https://mlsanity.com/paper/2101.12523/) (2021)
- [Overcoming Common Flaws in the Evaluation of Selective Classification Systems](https://mlsanity.com/paper/2407.01032/) (2024)
- [Data-IQ: Characterizing subgroups with heterogeneous outcomes in tabular data](https://mlsanity.com/paper/2210.13043/) (2022)
- [How to Evaluate Uncertainty Estimates in Machine Learning for Regression?](https://mlsanity.com/paper/2106.03395/) (2021)
- [Data-SUITE: Data-centric identification of in-distribution incongruous examples](https://mlsanity.com/paper/2202.08836/) (2022)
- [Dissecting Sample Hardness: A Fine-Grained Analysis of Hardness Characterization Methods for Data-Centric AI](https://mlsanity.com/paper/2403.04551/) (2024)
- [SCOD: From Heuristics to Theory](https://mlsanity.com/paper/2403.16916/) (2024)
- [PyHard: a novel tool for generating hardness embeddings to support data-centric analysis](https://mlsanity.com/paper/2109.14430/) (2021)
- [Characterizing instance hardness in classification and regression problems](https://mlsanity.com/paper/2212.01897/) (2022)

## Abstract (arXiv)

Machine Learning (ML) models are widely used in high-stakes domains such as healthcare, where the reliability of predictions is critical. However, these models often fail to account for uncertainty, providing predictions even with low confidence. This work proposes a novel two-step data-centric approach to enhance the performance of ML models by improving data quality and filtering low-confidence predictions. The first step involves leveraging Instance Hardness (IH) to filter problematic instances during training, thereby refining the dataset. The second step introduces a confidence-based rejection mechanism during inference, ensuring that only reliable predictions are retained. We evaluate our approach using three real-world healthcare datasets, demonstrating its effectiveness at improving model reliability while balancing predictive performance and rejection rate. Additionally, we use alternative criteria - influence values for filtering and uncertainty for rejection - as baselines to evaluate the efficiency of the proposed method. The results demonstrate that integrating IH filtering with confidence-based rejection effectively enhances model performance while preserving a large proportion of instances. This approach provides a practical method for deploying ML systems in safety-critical applications.