# Distributionally Robust Optimization with Bias and Variance Reduction

arXiv: https://arxiv.org/abs/2310.13863 · HTML: https://mlsanity.com/paper/2310.13863/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 109 references that are in the map.
- First posted on arXiv 21 Oct 2023.

## Builds on (most cited first, 1 of 1)

- [The iWildCam 2020 Competition Dataset](https://mlsanity.com/paper/2004.10340/) (2020)

## Built on by (most cited first, 6 of 6)

- [Efficient Algorithms for Empirical Group Distributionally Robust Optimization and Beyond](https://mlsanity.com/paper/2403.03562/) (2024)
- [Mitigating Spurious Correlation via Distributionally Robust Learning with Hierarchical Ambiguity Sets](https://mlsanity.com/paper/2510.02818/) (2025)
- [textttskwdro: a library for Wasserstein distributionally robust machine learning](https://mlsanity.com/paper/2410.21231/) (2024)
- [SOREL: A Stochastic Algorithm for Spectral Risks Minimization](https://mlsanity.com/paper/2407.14618/) (2024)
- [Certifiably Robust Model Evaluation in Federated Learning under Meta-Distributional Shifts](https://mlsanity.com/paper/2410.20250/) (2024)
- [Vision: Data-Centric Anchoring for Robust and Interpretable Agentic AI](https://mlsanity.com/paper/2609.08216/) (2026)

## Abstract (arXiv)

We consider the distributionally robust optimization (DRO) problem with spectral risk-based uncertainty set and f-divergence penalty. This formulation includes common risk-sensitive learning objectives such as regularized condition value-at-risk (CVaR) and average top-k loss. We present Prospect, a stochastic gradient-based algorithm that only requires tuning a single learning rate hyperparameter, and prove that it enjoys linear convergence for smooth regularized losses. This contrasts with previous algorithms that either require tuning multiple hyperparameters or potentially fail to converge due to biased gradient estimates or inadequate regularization. Empirically, we show that Prospect can converge 2-3× faster than baselines such as stochastic gradient and stochastic saddle-point methods on distribution shift and fairness benchmarks spanning tabular, vision, and language domains.