# Distributionally robust and generalizable inference

arXiv: https://arxiv.org/abs/2209.09352 · HTML: https://mlsanity.com/paper/2209.09352/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2022.

## Built on by (most cited first, 3 of 3)

- [Learning under random distributional shifts](https://mlsanity.com/paper/2306.02948/) (2023)
- [Distributionally robust risk evaluation with an isotonic constraint](https://mlsanity.com/paper/2407.06867/) (2024)
- [Efficient Multi-domain Text Recognition Deep Neural Network Parameterization with Residual Adapters](https://mlsanity.com/paper/2401.00971/) (2024)

## Abstract (arXiv)

We discuss recently developed methods that quantify the stability and generalizability of statistical findings under distributional changes. In many practical problems, the data is not drawn i.i.d. from the target population. For example, unobserved sampling bias, batch effects, or unknown associations might inflate the variance compared to i.i.d. sampling. For reliable statistical inference, it is thus necessary to account for these types of variation. We discuss and review two methods that allow quantifying distribution stability based on a single dataset. The first method computes the sensitivity of a parameter under worst-case distributional perturbations to understand which types of shift pose a threat to external validity. The second method treats distributional shifts as random which allows assessing average robustness (instead of worst-case). Based on a stability analysis of multiple estimators on a single dataset, it integrates both sampling and distributional uncertainty into a single confidence interval.