# Regularization via Mass Transportation

arXiv: https://arxiv.org/abs/1710.10016 · HTML: https://mlsanity.com/paper/1710.10016/

## Key facts

- Cited by 81 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Oct 2017.

## Built on by (most cited first, 30 of 81)

- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Recent Advances in Optimal Transport for Machine Learning](https://mlsanity.com/paper/2306.16156/) (2023)
- [Distributionally Robust Learning](https://mlsanity.com/paper/2108.08993/) (2021)
- [Sinkhorn Distributionally Robust Optimization](https://mlsanity.com/paper/2109.11926/) (2021)
- [Statistical Analysis of Wasserstein Distributionally Robust Estimators](https://mlsanity.com/paper/2108.02120/) (2021)
- [Fast Distributionally Robust Learning with Variance Reduced Min-Max Optimization](https://mlsanity.com/paper/2104.13326/) (2021)
- [Robust LLM Alignment via Distributionally Robust Direct Preference Optimization](https://mlsanity.com/paper/2502.01930/) (2025)
- [Distributionally Robust Bayesian Optimization with varphi-divergences](https://mlsanity.com/paper/2203.02128/) (2022)
- [Outlier-Robust Wasserstein DRO](https://mlsanity.com/paper/2311.05573/) (2023)
- [Semi-Discrete Optimal Transport: Hardness, Regularization and Numerical Solution](https://mlsanity.com/paper/2103.06263/) (2021)
- [Unifying Distributionally Robust Optimization via Optimal Transport Theory](https://mlsanity.com/paper/2308.05414/) (2023)
- [Nonsmooth Nonconvex-Nonconcave Minimax Optimization: Primal-Dual Balancing and Iteration Complexity Analysis](https://mlsanity.com/paper/2209.10825/) (2022)
- [Wasserstein Robust Classification with Fairness Constraints](https://mlsanity.com/paper/2103.06828/) (2021)
- [Generalization Bounds with Minimal Dependency on Hypothesis Class via Distributionally Robust Optimization](https://mlsanity.com/paper/2106.11180/) (2021)
- [Minimax risk classifiers with 0-1 loss](https://mlsanity.com/paper/2201.06487/) (2022)
- [Exact Generalization Guarantees for (Regularized) Wasserstein Distributionally Robust Models](https://mlsanity.com/paper/2305.17076/) (2023)
- [Nash Equilibria, Regularization and Computation in Optimal Transport-Based Distributionally Robust Optimization](https://mlsanity.com/paper/2303.03900/) (2023)
- [Certifying Out-of-Domain Generalization for Blackbox Functions](https://mlsanity.com/paper/2202.01679/) (2022)
- [Smoothed f-Divergence Distributionally Robust Optimization](https://mlsanity.com/paper/2306.14041/) (2023)
- [Robust Generalization despite Distribution Shift via Minimum Discriminating Information](https://mlsanity.com/paper/2106.04443/) (2021)
- [Tikhonov Regularization is Optimal Transport Robust under Martingale Constraints](https://mlsanity.com/paper/2210.01413/) (2022)
- [Coresets for Wasserstein Distributionally Robust Optimization Problems](https://mlsanity.com/paper/2210.04260/) (2022)
- [On Generalization and Regularization via Wasserstein Distributionally Robust Optimization](https://mlsanity.com/paper/2212.05716/) (2022)
- [Universal generalization guarantees for Wasserstein distributionally robust models](https://mlsanity.com/paper/2402.11981/) (2024)
- [Robust Distribution Learning with Local and Global Adversarial Corruptions](https://mlsanity.com/paper/2406.06509/) (2024)
- [DRO: A Python Library for Distributionally Robust Optimization in Machine Learning](https://mlsanity.com/paper/2505.23565/) (2025)
- [Adversarially Robust Kernel Smoothing](https://mlsanity.com/paper/2102.08474/) (2021)
- [Distributionally Robust Graph Learning from Smooth Signals under Moment Uncertainty](https://mlsanity.com/paper/2105.05458/) (2021)
- [On Certifying and Improving Generalization to Unseen Domains](https://mlsanity.com/paper/2206.12364/) (2022)
- [Learning Against Distributional Uncertainty: On the Trade-off Between Robustness and Specificity](https://mlsanity.com/paper/2301.13565/) (2023)

## Abstract (arXiv)

The goal of regression and classification methods in supervised learning is to minimize the empirical risk, that is, the expectation of some loss function quantifying the prediction error under the empirical distribution. When facing scarce training data, overfitting is typically mitigated by adding regularization terms to the objective that penalize hypothesis complexity. In this paper we introduce new regularization techniques using ideas from distributionally robust optimization, and we give new probabilistic interpretations to existing techniques. Specifically, we propose to minimize the worst-case expected loss, where the worst case is taken over the ball of all (continuous or discrete) distributions that have a bounded transportation distance from the (discrete) empirical distribution. By choosing the radius of this ball judiciously, we can guarantee that the worst-case expected loss provides an upper confidence bound on the loss on test data, thus offering new generalization bounds. We prove that the resulting regularized learning problems are tractable and can be tractably kernelized for many popular loss functions. We validate our theoretical out-of-sample guarantees through simulated and empirical experiments.