mlsanityData to 7 Oct 2026

Variance-based regularization with convex objectives

arXiv 1610.02581 ↗First posted 8 Oct 2016stat.ML math.ST stat.THcited by 153

Not on the map. Why?

Abstract (arXiv)

We develop an approach to risk minimization and stochastic optimization that provides a convex surrogate for variance, allowing near-optimal and computationally efficient trading between approximation and estimation error. Our approach builds off of techniques for distributionally robust optimization and Owen's empirical likelihood, and we provide a number of finite-sample and asymptotic results characterizing the theoretical performance of the estimator. In particular, we show that our procedure comes with certificates of optimality, achieving (in some scenarios) faster rates of convergence than empirical risk minimization by virtue of automatically balancing bias and variance. We give corroborating empirical evidence showing that in practice, the estimator indeed trades between variance and absolute performance on a training sample, improving out-of-sample (test) performance over standard empirical risk minimization for a number of classification problems.

Key facts. Cited by 153 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Oct 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 33; 2022: 30; 2023: 36; 2024: 23; 2025: 19; 2026: 11.

153 papers.

Sort:

By area:

  1. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  2. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimismcited by 275
  3. 2022Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlationscited by 171
  4. 2021Distributionally Robust Federated Averagingcited by 104
  5. 2021Risk-Averse Offline Reinforcement Learningcited by 65
  6. 2025Tapered Off-Policy REINFORCE: Stable and efficient reinforcement learning for LLMscited by 61
  7. 2024Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimizationcited by 52
  8. 2023Model Agnostic Sample Reweighting for Out-of-Distribution Learningcited by 43
  9. 2021On Tilted Losses in Machine Learning: Theory and Applicationscited by 42
  10. 2021Learning Fair Classifiers with Partially Annotated Group Labelscited by 40
  11. 2023Robust Learning with Progressive Data Expansion Against Spurious Correlationcited by 39
  12. 2022Federated Minimax Optimization: Improved Convergence Analyses and Algorithmscited by 37
38 more
  1. 2021Stability and Generalization of Stochastic Gradient Methods for Minimax Problemscited by 35
  2. 2022Doubly Robust Distributionally Robust Off-Policy Evaluation and Learningcited by 34
  3. 2022Policy learning "without" overlap: Pessimism and generalized empirical Bernstein's inequalitycited by 32
  4. 2022Learning to Solve Routing Problems via Distributionally Robust Optimizationcited by 31
  5. 2022Distributionally Robust Offline Reinforcement Learning with Linear Function Approximationcited by 31
  6. 2021Towards Theoretical Understandings of Robust Markov Decision Processes: Sample Complexity and Asymptoticscited by 27
  7. 2024C-RAG: Certified Generation Risks for Retrieval-Augmented Language Modelscited by 27
  8. 2022Rank-based Decomposable Losses in Machine Learning: A Surveycited by 26
  9. 2023Not All Semantics are Created Equal: Contrastive Self-supervised Learning with Automatic Temperature Individualizationcited by 25
  10. 2022Freeze then Train: Towards Provable Representation Learning under Spurious Correlations and Feature Noisecited by 22
  11. 2023Understanding Contrastive Learning via Distributionally Robust Optimizationcited by 21
  12. 2021Fast Distributionally Robust Learning with Variance Reduced Min-Max Optimizationcited by 19
  13. 2023Estimate-Then-Optimize versus Integrated-Estimation-Optimization versus Sample Average Approximation: A Stochastic Dominance Perspectivecited by 19
  14. 2025Learning from Heterogeneity: Generalizing Dynamic Facial Expression Recognition via Distributionally Robust Optimizationcited by 19
  15. 2021Distributionally Robust Multilingual Machine Translationcited by 18
  16. 2021Auxiliary Task Update Decomposition: The Good, The Bad and The Neutralcited by 17
  17. 2022Holistic Robust Data-Driven Decisionscited by 17
  18. 2022Distributionally Robust Bayesian Optimization with varphi-divergencescited by 16
  19. 2022Stochastic Constrained DRO with a Complexity Independent of Sample Sizecited by 16
  20. 2021Near Optimal Stochastic Algorithms for Finite-Sum Unbalanced Convex-Concave Minimax Optimizationcited by 15
  21. 2021Unified Group Fairness on Federated Learningcited by 15
  22. 2021Poisoning Attack against Estimating from Pairwise Comparisonscited by 14
  23. 2021Preventing dataset shift from breaking machine-learning biomarkerscited by 14
  24. 2022Distributionally Robust Optimization via Ball Oracle Accelerationcited by 14
  25. 2023Distributionally Robust Optimization with Probabilistic Groupcited by 14
  26. 2023Towards Artificial General Intelligence (AGI) in the Internet of Things (IoT): Opportunities and Challengescited by 14
  27. 2021Tilted Cross Entropy (TCE): Promoting Fairness in Semantic Segmentationcited by 12
  28. 2021Individually Fair Gradient Boostingcited by 12
  29. 2023Distributionally Robust Recourse Actioncited by 12
  30. 2023Unifying Distributionally Robust Optimization via Optimal Transport Theorycited by 12
  31. 2021Label Distributionally Robust Losses for Multi-class Classification: Consistency, Robustness and Adaptivitycited by 11
  32. 2022Certifying Some Distributional Fairness with Subpopulation Decompositioncited by 11
  33. 2023Robust Markov Decision Processes without Model Estimationcited by 11
  34. 2023Certified Robust Neural Networks: Generalization and Corruption Resistancecited by 11
  35. 2021The s-value: evaluating stability with respect to distributional shiftscited by 10
  36. 2021Quasi-Newton Methods for Saddle Point Problems and Beyondcited by 10
  37. 2024Bridging Multicalibration and Out-of-distribution Generalization Beyond Covariate Shiftcited by 10
  38. 2021Generalization Bounds with Minimal Dependency on Hypothesis Class via Distributionally Robust Optimizationcited by 9

50 most cited of 153 All 153 (JSON)