# Focus on the Common Good: Group Distributional Robustness Follows

arXiv: https://arxiv.org/abs/2110.02619 · HTML: https://mlsanity.com/paper/2110.02619/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 30 references that are in the map.
- First posted on arXiv 6 Oct 2021.

## Builds on (most cited first, 21 of 21)

- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Annotation Artifacts in Natural Language Inference Data](https://mlsanity.com/paper/1803.02324/) (2018)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification](https://mlsanity.com/paper/1903.04561/) (2019)
- [Fairness Without Demographics in Repeated Loss Minimization](https://mlsanity.com/paper/1806.08010/) (2018)
- [Functional Map of the World](https://mlsanity.com/paper/1711.07846/) (2017)
- [Learning Models with Uniform Performance via Distributionally Robust Optimization](https://mlsanity.com/paper/1810.08750/) (2018)
- [An Investigation of Why Overparameterization Exacerbates Spurious Correlations](https://mlsanity.com/paper/2005.04345/) (2020)
- [Environment Inference for Invariant Learning](https://mlsanity.com/paper/2010.07249/) (2020)
- [Generalizing Across Domains via Cross-Gradient Training](https://mlsanity.com/paper/1804.10745/) (2018)
- [Characterizing and Avoiding Negative Transfer](https://mlsanity.com/paper/1811.09751/) (2018)
- [Model Patching: Closing the Subgroup Performance Gap with Data Augmentation](https://mlsanity.com/paper/2008.06775/) (2020)
- [Examining and Combating Spurious Features under Distribution Shift](https://mlsanity.com/paper/2106.07171/) (2021)
- [Learning to select data for transfer learning with Bayesian Optimization](https://mlsanity.com/paper/1707.05246/) (2017)
- [DORO: Distributional and Outlier Robust Optimization](https://mlsanity.com/paper/2106.06142/) (2021)
- [Does Distributionally Robust Supervised Learning Give Robust Classifiers?](https://mlsanity.com/paper/1611.02041/) (2016)
- [On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment](https://mlsanity.com/paper/2010.03017/) (2020)
- [Predict then Interpolate: A Simple Algorithm to Learn Stable Classifiers](https://mlsanity.com/paper/2105.12628/) (2021)

## Built on by (most cited first, 25 of 25)

- [UMIX: Improving Importance Weighting for Subpopulation Shift via Uncertainty-Aware Mixup](https://mlsanity.com/paper/2209.08928/) (2022)
- [Robust Generalization against Photon-Limited Corruptions via Worst-Case Sharpness Minimization](https://mlsanity.com/paper/2303.13087/) (2023)
- [Robust Multi-Task Learning with Excess Risks](https://mlsanity.com/paper/2402.02009/) (2024)
- [Causal Balancing for Domain Generalization](https://mlsanity.com/paper/2206.05263/) (2022)
- [COCO-DR: Combating Distribution Shifts in Zero-Shot Dense Retrieval with Contrastive and Distributionally Robust Learning](https://mlsanity.com/paper/2210.15212/) (2022)
- [Domain Generalization via Rationale Invariance](https://mlsanity.com/paper/2308.11158/) (2023)
- [Distributionally Robust Post-hoc Classifiers under Prior Shifts](https://mlsanity.com/paper/2309.08825/) (2023)
- [AGRO: Adversarial Discovery of Error-prone groups for Robust Optimization](https://mlsanity.com/paper/2212.00921/) (2022)
- [Controllable Prompt Tuning For Balancing Group Distributional Robustness](https://mlsanity.com/paper/2403.02695/) (2024)
- [Intersectional Two-sided Fairness in Recommendation](https://mlsanity.com/paper/2402.02816/) (2024)
- [Mitigating Spurious Correlation via Distributionally Robust Learning with Hierarchical Ambiguity Sets](https://mlsanity.com/paper/2510.02818/) (2025)
- [Robust Distillation for Worst-class Performance](https://mlsanity.com/paper/2206.06479/) (2022)
- [Scaff-PD: Communication Efficient Fair and Robust Federated Learning](https://mlsanity.com/paper/2307.13381/) (2023)
- [Assessing and Enhancing Robustness of Deep Learning Models with Corruption Emulation in Digital Pathology](https://mlsanity.com/paper/2310.20427/) (2023)
- [Gated Domain Units for Multi-source Domain Generalization](https://mlsanity.com/paper/2206.12444/) (2022)
- [Class Distribution Shifts in Zero-Shot Learning: Learning Robust Representations](https://mlsanity.com/paper/2311.18575/) (2023)
- [A Causal Inspired Early-Branching Structure for Domain Generalization](https://mlsanity.com/paper/2403.08649/) (2024)
- [Use Perturbations when Learning from Explanations](https://mlsanity.com/paper/2303.06419/) (2023)
- [Reweighted Mixup for Subpopulation Shift](https://mlsanity.com/paper/2304.04148/) (2023)
- [Robust Representation Learning through Explicit Environment Modeling](https://mlsanity.com/paper/2604.26128/) (2026)
- [Improving Multitask Retrieval by Promoting Task Specialization](https://mlsanity.com/paper/2307.00342/) (2023)
- [Taking a Moment for Distributional Robustness](https://mlsanity.com/paper/2405.05461/) (2024)
- [Variational Learning of Disentangled Representations](https://mlsanity.com/paper/2506.17182/) (2025)
- [Environment-Robust Representation Learning with Empirical Bayes](https://mlsanity.com/paper/2606.05365/) (2026)
- [Learning the Robustness Mechanism with Bilevel Optimization](https://mlsanity.com/paper/2609.35541/) (2026)

## Abstract (arXiv)

We consider the problem of training a classification model with group annotated training data. Recent work has established that, if there is distribution shift across different groups, models trained using the standard empirical risk minimization (ERM) objective suffer from poor performance on minority groups and that group distributionally robust optimization (Group-DRO) objective is a better alternative. The starting point of this paper is the observation that though Group-DRO performs better than ERM on minority groups for some benchmark datasets, there are several other datasets where it performs much worse than ERM. Inspired by ideas from the closely related problem of domain generalization, this paper proposes a new and simple algorithm that explicitly encourages learning of features that are shared across various groups. The key insight behind our proposed algorithm is that while Group-DRO focuses on groups with worst regularized loss, focusing instead, on groups that enable better performance even on other groups, could lead to learning of shared/common features, thereby enhancing minority performance beyond what is achieved by Group-DRO. Empirically, we show that our proposed algorithm matches or achieves better performance compared to strong contemporary baselines including ERM and Group-DRO on standard benchmarks on both minority groups and across all groups. Theoretically, we show that the proposed algorithm is a descent method and finds first order stationary points of smooth nonconvex functions.