# Model Patching: Closing the Subgroup Performance Gap with Data Augmentation

arXiv: https://arxiv.org/abs/2008.06775 · HTML: https://mlsanity.com/paper/2008.06775/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Aug 2020.

## Built on by (most cited first, 30 of 83)

- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [Improving Out-of-Distribution Robustness via Selective Augmentation](https://mlsanity.com/paper/2201.00299/) (2022)
- [Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations](https://mlsanity.com/paper/2203.01517/) (2022)
- [A Fine-Grained Analysis on Distribution Shift](https://mlsanity.com/paper/2110.11328/) (2021)
- [Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspective](https://mlsanity.com/paper/2112.06409/) (2021)
- [ARGS: Alignment as Reward-Guided Search](https://mlsanity.com/paper/2402.01694/) (2024)
- [Process Reward Model with Q-Value Rankings](https://mlsanity.com/paper/2410.11287/) (2024)
- [BiaSwap: Removing dataset bias with bias-tailored swapping augmentation](https://mlsanity.com/paper/2108.10008/) (2021)
- [DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations](https://mlsanity.com/paper/2201.09637/) (2022)
- [Towards Last-layer Retraining for Group Robustness with Fewer Annotations](https://mlsanity.com/paper/2309.08534/) (2023)
- [Domain Generalization for Medical Image Analysis: A Review](https://mlsanity.com/paper/2310.08598/) (2023)
- [Out-of-distribution Generalization in the Presence of Nuisance-Induced Spurious Correlations](https://mlsanity.com/paper/2107.00520/) (2021)
- [Perfectly Balanced: Improving Transfer and Robustness of Supervised Contrastive Learning](https://mlsanity.com/paper/2204.07596/) (2022)
- [Robust Learning with Progressive Data Expansion Against Spurious Correlation](https://mlsanity.com/paper/2306.04949/) (2023)
- [Finding and Fixing Spurious Patterns with Explanations](https://mlsanity.com/paper/2106.02112/) (2021)
- [Representation Bias in Data: A Survey on Identification and Resolution Techniques](https://mlsanity.com/paper/2203.11852/) (2022)
- [Equitable modelling of brain imaging by counterfactual augmentation with morphologically constrained 3D deep generative models](https://mlsanity.com/paper/2111.14923/) (2021)
- [Are Vision Transformers Robust to Spurious Correlations?](https://mlsanity.com/paper/2203.09125/) (2022)
- [Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Data](https://mlsanity.com/paper/2310.16524/) (2023)
- [Calibrating Multi-modal Representations: A Pursuit of Group Robustness without Annotations](https://mlsanity.com/paper/2403.07241/) (2024)
- [Focus on the Common Good: Group Distributional Robustness Follows](https://mlsanity.com/paper/2110.02619/) (2021)
- [BARACK: Partially Supervised Group Robustness With Guarantees](https://mlsanity.com/paper/2201.00072/) (2021)
- [Robust Representation Learning via Perceptual Similarity Metrics](https://mlsanity.com/paper/2106.06620/) (2021)
- [Noise-robust Graph Learning by Estimating and Leveraging Pairwise Interactions](https://mlsanity.com/paper/2106.07451/) (2021)
- [Transferring Fairness under Distribution Shifts via Fair Consistency Regularization](https://mlsanity.com/paper/2206.12796/) (2022)
- [Efficient Conditionally Invariant Representation Learning](https://mlsanity.com/paper/2212.08645/) (2022)
- [How unfair is private learning ?](https://mlsanity.com/paper/2206.03985/) (2022)
- [Training Debiased Subnetworks with Contrastive Weight Pruning](https://mlsanity.com/paper/2210.05247/) (2022)
- [Distributionally Robust Optimization with Probabilistic Group](https://mlsanity.com/paper/2303.05809/) (2023)

## Abstract (arXiv)

Classifiers in machine learning are often brittle when deployed. Particularly concerning are models with inconsistent performance on specific subgroups of a class, e.g., exhibiting disparities in skin cancer classification in the presence or absence of a spurious bandage. To mitigate these performance differences, we introduce model patching, a two-stage framework for improving robustness that encourages the model to be invariant to subgroup differences, and focus on class information shared by subgroups. Model patching first models subgroup features within a class and learns semantic transformations between them, and then trains a classifier with data augmentations that deliberately manipulate subgroup features. We instantiate model patching with CAMEL, which (1) uses a CycleGAN to learn the intra-class, inter-subgroup augmentations, and (2) balances subgroup performance using a theoretically-motivated subgroup consistency regularizer, accompanied by a new robust objective. We demonstrate CAMEL's effectiveness on 3 benchmark datasets, with reductions in robust error of up to 33% relative to the best baseline. Lastly, CAMEL successfully patches a model that fails due to spurious features on a real-world skin cancer dataset.