# Distributionally Robust Losses for Latent Covariate Mixtures

arXiv: https://arxiv.org/abs/2007.13982 · HTML: https://mlsanity.com/paper/2007.13982/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2020.

## Built on by (most cited first, 30 of 58)

- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [NICO++: Towards Better Benchmarking for Domain Generalization](https://mlsanity.com/paper/2204.08040/) (2022)
- [The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models](https://mlsanity.com/paper/2107.00758/) (2021)
- [Unsupervised Learning of Debiased Representations with Pseudo-Attributes](https://mlsanity.com/paper/2108.02943/) (2021)
- [Perfectly Balanced: Improving Transfer and Robustness of Supervised Contrastive Learning](https://mlsanity.com/paper/2204.07596/) (2022)
- [Diagnosing Model Performance Under Distribution Shift](https://mlsanity.com/paper/2303.02011/) (2023)
- [Pareto Invariant Risk Minimization: Towards Mitigating the Optimization Dilemma in Out-of-Distribution Generalization](https://mlsanity.com/paper/2206.07766/) (2022)
- [An Online Learning Approach to Interpolation and Extrapolation in Domain Generalization](https://mlsanity.com/paper/2102.13128/) (2021)
- [Simple and near-optimal algorithms for hidden stratification and multi-group learning](https://mlsanity.com/paper/2112.12181/) (2021)
- [Regularizing towards Causal Invariance: Linear Models with Proxies](https://mlsanity.com/paper/2103.02477/) (2021)
- [A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalization](https://mlsanity.com/paper/2111.02355/) (2021)
- [Distributionally Robust Models with Parametric Likelihood Ratios](https://mlsanity.com/paper/2204.06340/) (2022)
- [Subgroup Robustness Grows On Trees: An Empirical Baseline Investigation](https://mlsanity.com/paper/2211.12703/) (2022)
- [Preventing dataset shift from breaking machine-learning biomarkers](https://mlsanity.com/paper/2107.09947/) (2021)
- [Distributionally Robust Survival Analysis: A Novel Fairness Loss Without Demographics](https://mlsanity.com/paper/2211.10508/) (2022)
- [Learning from a Biased Sample](https://mlsanity.com/paper/2209.01754/) (2022)
- [Fairness in Survival Analysis with Distributionally Robust Optimization](https://mlsanity.com/paper/2409.10538/) (2024)
- [Algorithmic Bias and Data Bias: Understanding the Relation between Distributionally Robust Optimization and Data Curation](https://mlsanity.com/paper/2106.09467/) (2021)
- [A comparison of approaches to improve worst-case predictive model performance over patient subpopulations](https://mlsanity.com/paper/2108.12250/) (2021)
- [Bridging Distributionally Robust Learning and Offline RL: An Approach to Mitigate Distribution Shift and Partial Data Coverage](https://mlsanity.com/paper/2310.18434/) (2023)
- [Addressing Polarization and Unfairness in Performative Prediction](https://mlsanity.com/paper/2406.16756/) (2024)
- [Task Generalization With AutoRegressive Compositional Structure: Can Learning From D Tasks Generalize to DT Tasks?](https://mlsanity.com/paper/2502.08991/) (2025)
- [Evaluating Robustness to Dataset Shift via Parametric Robustness Sets](https://mlsanity.com/paper/2205.15947/) (2022)
- [Understanding the effect of sparsity on neural networks robustness](https://mlsanity.com/paper/2206.10915/) (2022)
- [ExMap: Leveraging Explainability Heatmaps for Unsupervised Group Robustness to Spurious Correlations](https://mlsanity.com/paper/2403.13870/) (2024)
- [Learning Domain Invariant Representations by Joint Wasserstein Distance Minimization](https://mlsanity.com/paper/2106.04923/) (2021)
- [Scaff-PD: Communication Efficient Fair and Robust Federated Learning](https://mlsanity.com/paper/2307.13381/) (2023)
- [Is this model reliable for everyone? Testing for strong calibration](https://mlsanity.com/paper/2307.15247/) (2023)
- [Causality-oriented robustness: exploiting general noise interventions](https://mlsanity.com/paper/2307.10299/) (2023)
- [Geometry-Calibrated DRO: Combating Over-Pessimism with Free Energy Implications](https://mlsanity.com/paper/2311.05054/) (2023)

## Abstract (arXiv)

While modern large-scale datasets often consist of heterogeneous subpopulations -- for example, multiple demographic groups or multiple text corpora -- the standard practice of minimizing average loss fails to guarantee uniformly low losses across all subpopulations. We propose a convex procedure that controls the worst-case performance over all subpopulations of a given size. Our procedure comes with finite-sample (nonparametric) convergence guarantees on the worst-off subpopulation. Empirically, we observe on lexical similarity, wine quality, and recidivism prediction tasks that our worst-case procedure learns models that do well against unseen subpopulations.