# Distributionally Robust Optimization and Generalization in Kernel Methods

arXiv: https://arxiv.org/abs/1905.10943 · HTML: https://mlsanity.com/paper/1905.10943/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2019.

## Built on by (most cited first, 30 of 73)

- [Diagnosing Model Performance Under Distribution Shift](https://mlsanity.com/paper/2303.02011/) (2023)
- [Sinkhorn Distributionally Robust Optimization](https://mlsanity.com/paper/2109.11926/) (2021)
- [Learning to Augment Distributions for Out-of-Distribution Detection](https://mlsanity.com/paper/2311.01796/) (2023)
- [Regularized Policies are Reward Robust](https://mlsanity.com/paper/2101.07012/) (2021)
- [Minimax Regret Optimization for Robust Machine Learning under Distribution Shift](https://mlsanity.com/paper/2202.05436/) (2022)
- [Not All Semantics are Created Equal: Contrastive Self-supervised Learning with Automatic Temperature Individualization](https://mlsanity.com/paper/2305.11965/) (2023)
- [Near-Optimal Algorithms for Group Distributionally Robust Optimization and Beyond](https://mlsanity.com/paper/2212.13669/) (2022)
- [Distributionally Robust Bayesian Optimization with varphi-divergences](https://mlsanity.com/paper/2203.02128/) (2022)
- [Stochastic Constrained DRO with a Complexity Independent of Sample Size](https://mlsanity.com/paper/2210.05740/) (2022)
- [Domain-Specific Risk Minimization for Out-of-Distribution Generalization](https://mlsanity.com/paper/2208.08661/) (2022)
- [Score-based generative models are provably robust: an uncertainty quantification perspective](https://mlsanity.com/paper/2405.15754/) (2024)
- [Poisoning Attack against Estimating from Pairwise Comparisons](https://mlsanity.com/paper/2107.01854/) (2021)
- [Algorithmic Bias and Data Bias: Understanding the Relation between Distributionally Robust Optimization and Data Curation](https://mlsanity.com/paper/2106.09467/) (2021)
- [Instance-Dependent Generalization Bounds via Optimal Transport](https://mlsanity.com/paper/2211.01258/) (2022)
- [Distributionally Robust Reinforcement Learning with Human Feedback](https://mlsanity.com/paper/2503.00539/) (2025)
- [Generalization Bounds with Minimal Dependency on Hypothesis Class via Distributionally Robust Optimization](https://mlsanity.com/paper/2106.11180/) (2021)
- [Minimax risk classifiers with 0-1 loss](https://mlsanity.com/paper/2201.06487/) (2022)
- [Exact Generalization Guarantees for (Regularized) Wasserstein Distributionally Robust Models](https://mlsanity.com/paper/2305.17076/) (2023)
- [Moderately Distributional Exploration for Domain Generalization](https://mlsanity.com/paper/2304.13976/) (2023)
- [Optimal Control for Transformer Architectures: Enhancing Generalization, Robustness and Efficiency](https://mlsanity.com/paper/2505.13499/) (2025)
- [Prompt Disentanglement via Language Guidance and Representation Alignment for Domain Generalization](https://mlsanity.com/paper/2507.02288/) (2025)
- [Certifying Out-of-Domain Generalization for Blackbox Functions](https://mlsanity.com/paper/2202.01679/) (2022)
- [Online Kernel CUSUM for Change-Point Detection](https://mlsanity.com/paper/2211.15070/) (2022)
- [Adversarial Constrained Bidding via Minimax Regret Optimization with Causality-Aware Reinforcement Learning](https://mlsanity.com/paper/2306.07106/) (2023)
- [Smoothed f-Divergence Distributionally Robust Optimization](https://mlsanity.com/paper/2306.14041/) (2023)
- [Topology-aware Robust Optimization for Out-of-distribution Generalization](https://mlsanity.com/paper/2307.13943/) (2023)
- [On the Benefits of Over-parameterization for Out-of-Distribution Generalization](https://mlsanity.com/paper/2403.17592/) (2024)
- [Beyond the Federation: Topology-aware Federated Learning for Generalization to Unseen Clients](https://mlsanity.com/paper/2407.04949/) (2024)
- [Robust Generalization despite Distribution Shift via Minimum Discriminating Information](https://mlsanity.com/paper/2106.04443/) (2021)
- [Tikhonov Regularization is Optimal Transport Robust under Martingale Constraints](https://mlsanity.com/paper/2210.01413/) (2022)

## Abstract (arXiv)

Distributionally robust optimization (DRO) has attracted attention in machine learning due to its connections to regularization, generalization, and robustness. Existing work has considered uncertainty sets based on phi-divergences and Wasserstein distances, each of which have drawbacks. In this paper, we study DRO with uncertainty sets measured via maximum mean discrepancy (MMD). We show that MMD DRO is roughly equivalent to regularization by the Hilbert norm and, as a byproduct, reveal deep connections to classic results in statistical learning. In particular, we obtain an alternative proof of a generalization bound for Gaussian kernel ridge regression via a DRO lense. The proof also suggests a new regularizer. Our results apply beyond kernel methods: we derive a generically applicable approximation of MMD DRO, and show that it generalizes recent work on variance-based regularization.