# When Is a Conformal Guarantee Fair? Auditing Silent Subgroup Under-Coverage in Alzheimer's Disease Longitudinal Prediction

arXiv: https://arxiv.org/abs/2608.04254 · HTML: https://mlsanity.com/paper/2608.04254/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 31 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 11 of 11)

- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Adaptive Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2106.00170/) (2021)
- [Latent ODEs for Irregularly-Sampled Time Series](https://mlsanity.com/paper/1907.03907/) (2019)
- [Localized Conformal Prediction: A Generalized Inference Framework for Conformal Prediction](https://mlsanity.com/paper/2106.08460/) (2021)
- [Class-Conditional Conformal Prediction with Many Classes](https://mlsanity.com/paper/2306.09335/) (2023)
- [Copula-based conformal prediction for Multi-Target Regression](https://mlsanity.com/paper/2101.12002/) (2021)
- [A Unified Comparative Study with Generalized Conformity Scores for Multi-Output Conformal Regression](https://mlsanity.com/paper/2501.10533/) (2025)
- [Learning Spatio-Temporal Model of Disease Progression with NeuralODEs from Longitudinal Volumetric Data](https://mlsanity.com/paper/2211.04234/) (2022)
- [Uncertainty-Calibrated Prediction of Randomly-Timed Biomarker Trajectories with Conformal Bands](https://mlsanity.com/paper/2511.13911/) (2025)

## Abstract (arXiv)

Longitudinal prediction of Alzheimer's disease biomarkers increasingly informs clinical decisions, and a forecast is only useful if it also reports how much to trust it. Conformal prediction supplies this by wrapping any forecaster in a prediction band with a finite-sample coverage guarantee under exchangeability. However, standard population-level conformal prediction guarantees only marginal coverage and may mask substantial under-coverage within clinically important subgroups. We introduce a general mechanism-driven framework for auditing and repairing such subgroup under-coverage. Across two cohorts (ADNI, OASIS-3), two base forecasters, and nine attributes spanning genetic risk, demographics, and clinical severity, we find that population-level bands under-cover high-risk subgroups in 57 of 68 audited combinations, despite achieving nominal marginal coverage. We trace these failures to two mechanisms: (A) rarity, where a group-conditional band calibrated on only n patients covers at most k/(n+1); and (B) tail-heaviness, where a population-wide band is too narrow for a heavy-tailed subgroup and additional data cannot close the gap. Under-coverage falls disproportionately on patients with high genetic risk and disease severity (6.1 pp mean deficit, 95% CI [3.3, 8.9]), while demographic groups remain at the target level on average (0.0 pp, CI [-1.9, 1.7]). We pair each mechanism with a corresponding conformal correction: cross-conformal pooling for rarity, per-subgroup calibration for tail-heaviness, and a coverage-safe marginal floor when both arise. Together, these corrections restore target coverage for nearly every high-risk subgroup across both cohorts and forecasters.