# Beyond Metadata: CAPRA for Hidden Subgroup Analysis under Missing Metadata in Medical Imaging

arXiv: https://arxiv.org/abs/2607.09102 · HTML: https://mlsanity.com/paper/2607.09102/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 58 references that are in the map.
- First posted on arXiv 10 Jul 2026.

## Builds on (most cited first, 27 of 27)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs](https://mlsanity.com/paper/2303.00915/) (2023)
- [Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations](https://mlsanity.com/paper/2204.02937/) (2022)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging](https://mlsanity.com/paper/1909.12475/) (2019)
- [No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems](https://mlsanity.com/paper/2011.12945/) (2020)
- [BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks](https://mlsanity.com/paper/2305.17100/) (2023)
- [Domino: Discovering Systematic Errors with Cross-Modal Embeddings](https://mlsanity.com/paper/2203.14960/) (2022)
- [RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder](https://mlsanity.com/paper/2205.12035/) (2022)
- [A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervision](https://mlsanity.com/paper/2308.07898/) (2023)
- [FairCLIP: Harnessing Fairness in Vision-Language Learning](https://mlsanity.com/paper/2403.19949/) (2024)
- [MEDFAIR: Benchmarking Fairness for Medical Imaging](https://mlsanity.com/paper/2210.01725/) (2022)
- [Simple and Fast Group Robustness by Automatic Feature Reweighting](https://mlsanity.com/paper/2306.11074/) (2023)
- [Towards Last-layer Retraining for Group Robustness with Fewer Annotations](https://mlsanity.com/paper/2309.08534/) (2023)
- [Deep Learning Predicts Hip Fracture using Confounding Patient and Healthcare Variables](https://mlsanity.com/paper/1811.03695/) (2018)
- [T2-GNN: Graph Neural Networks for Graphs with Incomplete Features and Structure via Teacher-Student Distillation](https://mlsanity.com/paper/2212.12738/) (2022)
- [Simple and near-optimal algorithms for hidden stratification and multi-group learning](https://mlsanity.com/paper/2112.12181/) (2021)
- [Addressing Fairness Issues in Deep Learning-Based Medical Image Analysis: A Systematic Review](https://mlsanity.com/paper/2209.13177/) (2022)
- [Slicing Through Bias: Explaining Performance Gaps in Medical Image Analysis using Slice Discovery Methods](https://mlsanity.com/paper/2406.12142/) (2024)
- [Deep Learning Discovery of Demographic Biomarkers in Echocardiography](https://mlsanity.com/paper/2207.06421/) (2022)
- [ExMap: Leveraging Explainability Heatmaps for Unsupervised Group Robustness to Spurious Correlations](https://mlsanity.com/paper/2403.13870/) (2024)
- [Group-robust Sample Reweighting for Subpopulation Shifts via Influence Functions](https://mlsanity.com/paper/2503.07315/) (2025)
- [Diverse Prototypical Ensembles Improve Robustness to Subpopulation Shift](https://mlsanity.com/paper/2505.23027/) (2025)
- [Subgroup Performance Analysis in Hidden Stratifications](https://mlsanity.com/paper/2503.10382/) (2025)
- [Leveraging Vision-Language Foundation Models to Reveal Hidden Image-Attribute Relationships in Medical Imaging](https://mlsanity.com/paper/2503.23618/) (2025)

## Abstract (arXiv)

Medical imaging models are often deployed without the demographic, acquisition, and quality metadata needed for subgroup auditing. Once those metadata disappear, clinically critical failure modes can be masked by strong aggregate performance, and many robust-learning methods lose the group structure they rely on. We present CAPRA, a calibrated proxy-axis framework for hidden subgroup analysis under missing metadata. CAPRA predicts image-derived semantic axes, calibrates axis posteriors on a small metadata-labeled split via patient-level cross-fitting, and organizes those posteriors into a calibrated subgroup interface that supports both deployment-time failure analysis and downstream robust learning without requiring subgroup labels at deployment. Across fundus, dermoscopy, and chest radiography, CAPRA reveals disparity patterns missed by metadata-only slicing, remains informative under dataset shift, and produces subgroup partitions that align more closely with explicit failure axes than image-only or latent-slice baselines. The same interface can also be reused by downstream robust learners, although those gains are domain-dependent. Overall, CAPRA turns hidden subgroup analysis under missing metadata into a calibrated, interpretable, and reusable subgroup interface for deployment-time analysis and robust transfer.