mlsanityData to 14 Oct 2026

Large-Scale Methods for Distributionally Robust Optimization

arXiv 2010.05893 ↗First posted 12 Oct 2020math.OC cs.LG stat.MLcited by 179

Not on the map. Why?

Abstract (arXiv)

We propose and analyze algorithms for distributionally robust optimization of convex losses with conditional value at risk (CVaR) and chi2 divergence uncertainty sets. We prove that our algorithms require a number of gradient evaluations independent of training set size and number of parameters, making them suitable for large-scale applications. For chi2 uncertainty sets these are the first such guarantees in the literature, and for CVaR our guarantees scale linearly in the uncertainty level rather than quadratically as in previous work. We also provide lower bounds proving the worst-case optimality of our algorithms for CVaR and a penalized version of the chi2 problem. Our primary technical contributions are novel bounds on the bias of batch robust risk estimation and the variance of a multilevel Monte Carlo gradient estimator due to [Blanchet & Glynn, 2015]. Experiments on MNIST and ImageNet confirm the theoretical scaling of our algorithms, which are 9--36 times more efficient than full-batch methods.

Key facts. Cited by 179 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 17; 2022: 38; 2023: 37; 2024: 39; 2025: 25; 2026: 23.

179 papers.

Sort:

By area:

  1. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 523
  2. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 436
  3. 2022Challenges and Strategies in Cross-Cultural NLPcited by 218
  4. 2022Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlationscited by 172
  5. 2024RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Foldcited by 99
  6. 2022Spread Spurious Attribute: Improving Worst-group Accuracy with Spurious Attribute Estimationcited by 88
  7. 2023DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedulecited by 85
  8. 2023Discover and Cure: Concept-aware Mitigation of Spurious Correlationcited by 71
  9. 2024Preserving Fairness Generalization in Deepfake Detectioncited by 70
  10. 2022MaskTune: Mitigating Spurious Correlations by Forcing to Explorecited by 66
  11. 2023Benchmarking Distribution Shift in Tabular Data with TableShiftcited by 66
  12. 2021Risk-Averse Offline Reinforcement Learningcited by 65
38 more
  1. 2022Diversify and Disambiguate: Learning From Underspecified Datacited by 56
  2. 2022Learning Debiased Classifier with Biased Committeecited by 51
  3. 2023Model Agnostic Sample Reweighting for Out-of-Distribution Learningcited by 44
  4. 2021On Tilted Losses in Machine Learning: Theory and Applicationscited by 43
  5. 2024The Clever Hans Mirage: A Comprehensive Survey on Spurious Correlations in Machine Learningcited by 41
  6. 2023A Guide Through the Zoo of Biased SGDcited by 40
  7. 2023Improving Fairness in Deepfake Detectioncited by 40
  8. 2021Non-convex Distributionally Robust Optimization: Non-asymptotic Analysiscited by 39
  9. 2021Sinkhorn Distributionally Robust Optimizationcited by 38
  10. 2022Adapting to Mixing Time in Stochastic Optimization with Markovian Datacited by 36
  11. 2025Representation Learning for Tabular Data: A Comprehensive Surveycited by 36
  12. 2022UMIX: Improving Importance Weighting for Subpopulation Shift via Uncertainty-Aware Mixupcited by 34
  13. 2022Calibrated Selective Classificationcited by 33
  14. 2022Learning Fair Representation via Distributional Contrastive Disentanglementcited by 31
  15. 2022When AUC meets DRO: Optimizing Partial AUC for Deep Learning with Non-Convex Convergence Guaranteecited by 29
  16. 2024AI-Face: A Million-Scale Demographically Annotated AI-Generated Face Dataset and Fairness Benchmarkcited by 28
  17. 2022Rank-based Decomposable Losses in Machine Learning: A Surveycited by 26
  18. 2021Stochastic Bias-Reduced Gradient Methodscited by 25
  19. 2021BARACK: Partially Supervised Group Robustness With Guaranteescited by 25
  20. 2023Not All Semantics are Created Equal: Contrastive Self-supervised Learning with Automatic Temperature Individualizationcited by 25
  21. 2023Improving Generalization of Alignment with Human Preferences through Group Invariant Learningcited by 25
  22. 2021Distributionally Robust Multilingual Machine Translationcited by 18
  23. 2023On the Theories Behind Hard Negative Sampling for Recommendationcited by 18
  24. 2025Robust LLM Alignment via Distributionally Robust Direct Preference Optimizationcited by 18
  25. 2022Distributionally Robust Models with Parametric Likelihood Ratioscited by 17
  26. 2022Subgroup Robustness Grows On Trees: An Empirical Baseline Investigationcited by 17
  27. 2022Learning to Split for Automatic Bias Detectioncited by 16
  28. 2022Stochastic Constrained DRO with a Complexity Independent of Sample Sizecited by 16
  29. 2025Upweighting Easy Samples in Fine-Tuning Mitigates Forgettingcited by 16
  30. 2024A Short Survey on Importance Weighting for Machine Learningcited by 15
  31. 2022Distributionally Robust Optimization via Ball Oracle Accelerationcited by 14
  32. 2022Learning from a Biased Samplecited by 14
  33. 2022Distributed Distributionally Robust Optimization with Non-Convex Objectivescited by 14
  34. 2023Distributionally Robust Optimization with Probabilistic Groupcited by 14
  35. 2025Sign Operator for Coping with Heavy-Tailed Noise in Non-Convex Optimization: High Probability Bounds Under (L0, L1)-Smoothnesscited by 13
  36. 2021Constrained Instance and Class Reweighting for Robust Learning under Label Noisecited by 12
  37. 2022Nonsmooth Nonconvex-Nonconcave Minimax Optimization: Primal-Dual Balancing and Iteration Complexity Analysiscited by 12
  38. 2022Universal Gradient Descent Ascent Method for Nonconvex-Nonconcave Minimax Optimizationcited by 12

50 most cited of 179 All 179 (JSON)