# No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems

arXiv: https://arxiv.org/abs/2011.12945 · HTML: https://mlsanity.com/paper/2011.12945/

## Key facts

- Cited by 209 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2020.

## Built on by (most cited first, 30 of 209)

- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision](https://mlsanity.com/paper/2312.09390/) (2023)
- [Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations](https://mlsanity.com/paper/2204.02937/) (2022)
- [Data-centric Artificial Intelligence: A Survey](https://mlsanity.com/paper/2303.10158/) (2023)
- [Simple data balancing achieves competitive worst-group-accuracy](https://mlsanity.com/paper/2110.14503/) (2021)
- [Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations](https://mlsanity.com/paper/2203.01517/) (2022)
- [Domino: Discovering Systematic Errors with Cross-Modal Embeddings](https://mlsanity.com/paper/2203.14960/) (2022)
- [Auditing large language models: a three-layered approach](https://mlsanity.com/paper/2302.08500/) (2023)
- [Model-Based Domain Generalization](https://mlsanity.com/paper/2102.11436/) (2021)
- [Spread Spurious Attribute: Improving Worst-group Accuracy with Spurious Attribute Estimation](https://mlsanity.com/paper/2204.02070/) (2022)
- [A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Others](https://mlsanity.com/paper/2212.04825/) (2022)
- [Distilling Model Failures as Directions in Latent Space](https://mlsanity.com/paper/2206.14754/) (2022)
- [Discover and Cure: Concept-aware Mitigation of Spurious Correlation](https://mlsanity.com/paper/2305.00650/) (2023)
- [Simple and Fast Group Robustness by Automatic Feature Reweighting](https://mlsanity.com/paper/2306.11074/) (2023)
- [MaskTune: Mitigating Spurious Correlations by Forcing to Explore](https://mlsanity.com/paper/2210.00055/) (2022)
- [Examining and Combating Spurious Features under Distribution Shift](https://mlsanity.com/paper/2106.07171/) (2021)
- [The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models](https://mlsanity.com/paper/2107.00758/) (2021)
- [Contrastive Adapters for Foundation Model Group Robustness](https://mlsanity.com/paper/2207.07180/) (2022)
- [ZIN: When and How to Learn Invariance Without Environment Partition?](https://mlsanity.com/paper/2203.05818/) (2022)
- [Discovering and Mitigating Visual Biases through Keyword Explanation](https://mlsanity.com/paper/2301.11104/) (2023)
- [Diagnosing and Rectifying Vision Models using Language](https://mlsanity.com/paper/2302.04269/) (2023)
- [Towards Last-layer Retraining for Group Robustness with Fewer Annotations](https://mlsanity.com/paper/2309.08534/) (2023)
- [Learning Debiased Classifier with Biased Committee](https://mlsanity.com/paper/2206.10843/) (2022)
- [Discover and Mitigate Unknown Biases with Debiasing Alternate Networks](https://mlsanity.com/paper/2207.10077/) (2022)
- [Unsupervised Learning of Debiased Representations with Pseudo-Attributes](https://mlsanity.com/paper/2108.02943/) (2021)
- [Identifying Spurious Biases Early in Training through the Lens of Simplicity Bias](https://mlsanity.com/paper/2305.18761/) (2023)
- [Spawrious: A Benchmark for Fine Control of Spurious Correlation Biases](https://mlsanity.com/paper/2303.05470/) (2023)
- [Perfectly Balanced: Improving Transfer and Robustness of Supervised Contrastive Learning](https://mlsanity.com/paper/2204.07596/) (2022)
- [Adaptive Testing of Computer Vision Models](https://mlsanity.com/paper/2212.02774/) (2022)
- [Robust Learning with Progressive Data Expansion Against Spurious Correlation](https://mlsanity.com/paper/2306.04949/) (2023)

## Abstract (arXiv)

In real-world classification tasks, each class often comprises multiple finer-grained "subclasses." As the subclass labels are frequently unavailable, models trained using only the coarser-grained class labels often exhibit highly variable performance across different subclasses. This phenomenon, known as hidden stratification, has important consequences for models deployed in safety-critical applications such as medicine. We propose GEORGE, a method to both measure and mitigate hidden stratification even when subclass labels are unknown. We first observe that unlabeled subclasses are often separable in the feature space of deep neural networks, and exploit this fact to estimate subclass labels for the training data via clustering techniques. We then use these approximate subclass labels as a form of noisy supervision in a distributionally robust optimization objective. We theoretically characterize the performance of GEORGE in terms of the worst-case generalization error across any subclass. We empirically validate GEORGE on a mix of real-world and benchmark image classification datasets, and show that our approach boosts worst-case subclass accuracy by up to 22 percentage points compared to standard training techniques, without requiring any prior information about the subclasses.