# An Empirical Analysis of Calibration and Selective Prediction in Multimodal Clinical Condition Classification

arXiv: https://arxiv.org/abs/2603.02719 · HTML: https://mlsanity.com/paper/2603.02719/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 41 references that are in the map.
- First posted on arXiv 3 Mar 2026.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [MedFuse: Multi-modal fusion with clinical time-series data and chest X-ray images](https://mlsanity.com/paper/2207.07027/) (2022)
- [DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistency](https://mlsanity.com/paper/2403.06197/) (2024)
- [Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation](https://mlsanity.com/paper/2403.08002/) (2024)
- [Function-Space Regularization in Neural Networks: A Probabilistic Perspective](https://mlsanity.com/paper/2312.17162/) (2023)
- [Has Multimodal Learning Delivered Universal Intelligence in Healthcare? A Comprehensive Survey](https://mlsanity.com/paper/2408.12880/) (2024)
- [Multimodal Machine Learning in Image-Based and Clinical Biomedicine: Survey and Prospects](https://mlsanity.com/paper/2311.02332/) (2023)
- [A Survey on Safe Multi-Modal Learning System](https://mlsanity.com/paper/2402.05355/) (2024)
- [Should We Learn Most Likely Functions or Parameters?](https://mlsanity.com/paper/2311.15990/) (2023)
- [What Does It Take to Build a Performant Selective Classifier?](https://mlsanity.com/paper/2510.20242/) (2025)

## Built on by (most cited first, 2 of 2)

- [The Large Cancer Assistant (LCA): A Model-Agnostic Orchestration Framework for Scalable Clinical Decision Support in Oncology](https://mlsanity.com/paper/2607.06531/) (2026)
- [OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System One](https://mlsanity.com/paper/2610.00381/) (2026)

## Abstract (arXiv)

As artificial intelligence systems move toward clinical deployment, ensuring reliable prediction behavior is fundamental for safety-critical decision-making tasks. One proposed safeguard is selective prediction, where models can defer uncertain predictions to human experts for review. In this work, we empirically evaluate the reliability of uncertainty-based selective prediction in multilabel clinical condition classification using multimodal ICU data. Across a range of state-of-the-art unimodal and multimodal models, we find that selective prediction can substantially degrade performance despite strong standard evaluation metrics. This failure is driven by severe class-dependent miscalibration, whereby models assign high uncertainty to correct predictions and low uncertainty to incorrect ones, particularly for underrepresented clinical conditions. Our results show that commonly used aggregate metrics can obscure these effects, limiting their ability to assess selective prediction behavior in this setting. Taken together, our findings characterize a task-specific failure mode of selective prediction in multimodal clinical condition classification and highlight the need for calibration-aware evaluation to provide strong guarantees of safety and robustness in clinical AI.