# Learning Against Distributional Uncertainty: On the Trade-off Between Robustness and Specificity

arXiv: https://arxiv.org/abs/2301.13565 · HTML: https://mlsanity.com/paper/2301.13565/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 71 references that are in the map.
- First posted on arXiv 31 Jan 2023.

## Builds on (most cited first, 23 of 23)

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Information-theoretic analysis of generalization capability of learning algorithms](https://mlsanity.com/paper/1705.07809/) (2017)
- [Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations](https://mlsanity.com/paper/1505.05116/) (2015)
- [Distributionally Robust Stochastic Optimization with Wasserstein Distance](https://mlsanity.com/paper/1604.02199/) (2016)
- [Large-Scale Methods for Distributionally Robust Optimization](https://mlsanity.com/paper/2010.05893/) (2020)
- [Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning](https://mlsanity.com/paper/1908.08729/) (2019)
- [Adversarial Training Can Hurt Generalization](https://mlsanity.com/paper/1906.06032/) (2019)
- [PAC-Bayesian Theory Meets Bayesian Inference](https://mlsanity.com/paper/1605.08636/) (2016)
- [Regularization via Mass Transportation](https://mlsanity.com/paper/1710.10016/) (2017)
- [Distributionally Robust Optimization: A Review](https://mlsanity.com/paper/1908.05659/) (2019)
- [Finite-Sample Guarantees for Wasserstein Distributionally Robust Optimization: Breaking the Curse of Dimensionality](https://mlsanity.com/paper/2009.04382/) (2020)
- [Distributionally Robust Learning](https://mlsanity.com/paper/2108.08993/) (2021)
- [Cross-validation failure: small sample sizes lead to large error bars](https://mlsanity.com/paper/1706.07581/) (2017)
- [Tighter expected generalization error bounds via Wasserstein distance](https://mlsanity.com/paper/2101.09315/) (2021)
- [Statistical Analysis of Wasserstein Distributionally Robust Estimators](https://mlsanity.com/paper/2108.02120/) (2021)
- [On Linear Optimization over Wasserstein Balls](https://mlsanity.com/paper/2004.07162/) (2020)
- [Concentration Inequalities for Statistical Inference](https://mlsanity.com/paper/2011.02258/) (2020)
- [When can we improve on sample average approximation for stochastic optimization?](https://mlsanity.com/paper/1907.08334/) (2019)
- [Distributional Robustness Bounds Generalization Errors](https://mlsanity.com/paper/2212.09962/) (2022)

## Built on by (most cited first, 4 of 4)

- [Decision Making under the Exponential Family: Distributionally Robust Optimisation with Bayesian Ambiguity Sets](https://mlsanity.com/paper/2411.16829/) (2024)
- [Decision Making under Model Misspecification: DRO with Robust Bayesian Ambiguity Sets](https://mlsanity.com/paper/2505.03585/) (2025)
- [Robust Processing and Learning: Principles, Methods, and Wireless Applications](https://mlsanity.com/paper/2602.09848/) (2026)
- [Conservative neural posterior estimation via distributionally robust training](https://mlsanity.com/paper/2605.28516/) (2026)

## Abstract (arXiv)

Trustworthy machine learning aims at combating distributional uncertainties in training data distributions compared to population distributions. Typical treatment frameworks include the Bayesian approach, (min-max) distributionally robust optimization (DRO), and regularization. However, three issues have to be raised: 1) the prior distribution in the Bayesian method and the regularizer in the regularization method are difficult to specify; 2) the DRO method tends to be overly conservative; 3) all the three methods are biased estimators of the true optimal cost. This paper studies a new framework that unifies the three approaches and addresses the three challenges above. The asymptotic properties (e.g., consistencies and asymptotic normalities), non-asymptotic properties (e.g., generalization bounds and unbiasedness), and solution methods of the proposed model are studied. The new model reveals the trade-off between the robustness to the unseen data and the specificity to the training data. Experiments on various real-world tasks validate the superiority of the proposed learning framework.