# Invariant Probabilistic Prediction

arXiv: https://arxiv.org/abs/2309.10083 · HTML: https://mlsanity.com/paper/2309.10083/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 41 references that are in the map.
- First posted on arXiv 18 Sep 2023.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Learning Models with Uniform Performance via Distributionally Robust Optimization](https://mlsanity.com/paper/1810.08750/) (2018)
- [Invariance, Causality and Robustness](https://mlsanity.com/paper/1812.08233/) (2018)
- [Anchor regression: heterogeneous data meets causality](https://mlsanity.com/paper/1801.06229/) (2018)
- [Robust Validation: Confident Predictions Even When Distributions Shift](https://mlsanity.com/paper/2008.04267/) (2020)
- [A causal framework for distribution generalization](https://mlsanity.com/paper/2006.07433/) (2020)
- [backShift: Learning causal cyclic graphs from unknown shift interventions](https://mlsanity.com/paper/1506.02494/) (2015)
- [Distributionally Robust Local Non-parametric Conditional Estimation](https://mlsanity.com/paper/2010.05373/) (2020)
- [Distributional robustness of K-class estimators and the PULSE](https://mlsanity.com/paper/2005.03353/) (2020)
- [Distributional Anchor Regression](https://mlsanity.com/paper/2101.08224/) (2021)
- [Causal Dantzig: fast inference in linear structural equation models with hidden variables under additive interventions](https://mlsanity.com/paper/1706.06159/) (2017)
- [Local scale invariance and robustness of proper scoring rules](https://mlsanity.com/paper/1912.05642/) (2019)
- [Distributional (Single) Index Models](https://mlsanity.com/paper/2006.09219/) (2020)

## Abstract (arXiv)

In recent years, there has been a growing interest in statistical methods that exhibit robust performance under distribution changes between training and test data. While most of the related research focuses on point predictions with the squared error loss, this article turns the focus towards probabilistic predictions, which aim to comprehensively quantify the uncertainty of an outcome variable given covariates. Within a causality-inspired framework, we investigate the invariance and robustness of probabilistic predictions with respect to proper scoring rules. We show that arbitrary distribution shifts do not, in general, admit invariant and robust probabilistic predictions, in contrast to the setting of point prediction. We illustrate how to choose evaluation metrics and restrict the class of distribution shifts to allow for identifiability and invariance in the prototypical Gaussian heteroscedastic linear model. Motivated by these findings, we propose a method to yield invariant probabilistic predictions, called IPP, and study the consistency of the underlying parameters. Finally, we demonstrate the empirical performance of our proposed procedure on simulated as well as on single-cell data.