# Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation

arXiv: https://arxiv.org/abs/1910.10583 · HTML: https://mlsanity.com/paper/1910.10583/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2019.

## Built on by (most cited first, 13 of 13)

- [Statistical Analysis of Wasserstein Distributionally Robust Estimators](https://mlsanity.com/paper/2108.02120/) (2021)
- [Robust Bayesian Recourse](https://mlsanity.com/paper/2206.10833/) (2022)
- [Sequential Domain Adaptation by Synthesizing Distributionally Robust Experts](https://mlsanity.com/paper/2106.00322/) (2021)
- [Efficient Stochastic Approximation of Minimax Excess Risk Optimization](https://mlsanity.com/paper/2306.00026/) (2023)
- [A data-driven approach to beating SAA out-of-sample](https://mlsanity.com/paper/2105.12342/) (2021)
- [Rockafellian Relaxation and Stochastic Optimization under Perturbations](https://mlsanity.com/paper/2204.04762/) (2022)
- [A Geometric Unification of Distributionally Robust Covariance Estimators: Shrinking the Spectrum by Inflating the Ambiguity Set](https://mlsanity.com/paper/2405.20124/) (2024)
- [A Greedy and Optimistic Approach to Clustering with a Specified Uncertainty of Covariates](https://mlsanity.com/paper/2204.08205/) (2022)
- [On Tractability, Complexity, and Mixed-Integer Convex Programming Representability of Distributionally Favorable Optimization](https://mlsanity.com/paper/2401.17899/) (2024)
- [Mitigating the Impact of Labeling Errors on Training via Rockafellian Relaxation](https://mlsanity.com/paper/2405.20531/) (2024)
- [Zeroth-Order Stochastic Mirror Descent Algorithms for Minimax Excess Risk Optimization](https://mlsanity.com/paper/2408.12209/) (2024)
- [An integrated perspective of robustness in regression through the lens of the bias-variance trade-off](https://mlsanity.com/paper/2407.10418/) (2024)
- [Breaking the Curse of Repulsion: Remoteness-Aware Control of Negative Off-Policy Updates](https://mlsanity.com/paper/2602.10430/) (2026)

## Abstract (arXiv)

The likelihood function is a fundamental component in Bayesian statistics. However, evaluating the likelihood of an observation is computationally intractable in many applications. In this paper, we propose a non-parametric approximation of the likelihood that identifies a probability measure which lies in the neighborhood of the nominal measure and that maximizes the probability of observing the given sample point. We show that when the neighborhood is constructed by the Kullback-Leibler divergence, by moment conditions or by the Wasserstein distance, then our textitoptimistic likelihood can be determined through the solution of a convex optimization problem, and it admits an analytical expression in particular cases. We also show that the posterior inference problem with our optimistic likelihood approximation enjoys strong theoretical performance guarantees, and it performs competitively in a probabilistic classification task.