# Subgroup Performance Analysis in Hidden Stratifications

arXiv: https://arxiv.org/abs/2503.10382 · HTML: https://mlsanity.com/paper/2503.10382/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 13 Mar 2025.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging](https://mlsanity.com/paper/1909.12475/) (2019)
- [Domino: Discovering Systematic Errors with Cross-Modal Embeddings](https://mlsanity.com/paper/2203.14960/) (2022)
- [CheXpert Plus: Augmenting a Large Chest X-ray Dataset with Text Radiology Reports, Patient Demographics and Additional Image Formats](https://mlsanity.com/paper/2405.19538/) (2024)
- [The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models](https://mlsanity.com/paper/2107.00758/) (2021)
- [Debiasing Skin Lesion Datasets and Models? Not So Fast](https://mlsanity.com/paper/2004.11457/) (2020)
- [FACTS: First Amplify Correlations and Then Slice to Discover Bias](https://mlsanity.com/paper/2309.17430/) (2023)
- [Detecting Shortcuts in Medical Images -- A Case Study in Chest X-rays](https://mlsanity.com/paper/2211.04279/) (2022)
- [Right for the Wrong Reason: Can Interpretable ML Techniques Detect Spurious Correlations?](https://mlsanity.com/paper/2307.12344/) (2023)
- [BiasPruner: Debiased Continual Learning for Medical Image Classification](https://mlsanity.com/paper/2407.08609/) (2024)
- [Confidence intervals uncovered: Are we ready for real-world medical imaging AI?](https://mlsanity.com/paper/2409.17763/) (2024)
- [Even Small Correlation and Diversity Shifts Pose Dataset-Bias Issues](https://mlsanity.com/paper/2305.05807/) (2023)

## Built on by (most cited first, 3 of 3)

- [A multimodal slice discovery framework for systematic failure detection and explanation in medical image classification](https://mlsanity.com/paper/2602.24183/) (2026)
- [Fairness Beyond Demographics: Optimizing Performance Across Appearance-Based Hidden Cohorts in Medical Imaging](https://mlsanity.com/paper/2605.29827/) (2026)
- [Beyond Metadata: CAPRA for Hidden Subgroup Analysis under Missing Metadata in Medical Imaging](https://mlsanity.com/paper/2607.09102/) (2026)

## Abstract (arXiv)

Machine learning (ML) models may suffer from significant performance disparities between patient groups. Identifying such disparities by monitoring performance at a granular level is crucial for safely deploying ML to each patient. Traditional subgroup analysis based on metadata can expose performance disparities only if the available metadata (e.g., patient sex) sufficiently reflects the main reasons for performance variability, which is not common. Subgroup discovery techniques that identify cohesive subgroups based on learned feature representations appear as a potential solution: They could expose hidden stratifications and provide more granular subgroup performance reports. However, subgroup discovery is challenging to evaluate even as a standalone task, as ground truth stratification labels do not exist in real data. Subgroup discovery has thus neither been applied nor evaluated for the application of subgroup performance monitoring. Here, we apply subgroup discovery for performance monitoring in chest x-ray and skin lesion classification. We propose novel evaluation strategies and show that a simplified subgroup discovery method without access to classification labels or metadata can expose larger performance disparities than traditional metadata-based subgroup analysis. We provide the first compelling evidence that subgroup discovery can serve as an important tool for comprehensive performance validation and monitoring of trustworthy AI in medicine.