# Bayesian Invariance Modeling of Multi-Environment Data

arXiv: https://arxiv.org/abs/2506.22675 · HTML: https://mlsanity.com/paper/2506.22675/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 43 references that are in the map.
- First posted on arXiv 27 Jun 2025.

## Builds on (most cited first, 16 of 16)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [Monte Carlo Gradient Estimation in Machine Learning](https://mlsanity.com/paper/1906.10652/) (2019)
- [Invariant Causal Prediction for Nonlinear Models](https://mlsanity.com/paper/1706.08576/) (2017)
- [Invariance, Causality and Robustness](https://mlsanity.com/paper/1812.08233/) (2018)
- [Probabilistic Best Subset Selection via Gradient-Based Optimization](https://mlsanity.com/paper/2006.06448/) (2020)
- [Environment Invariant Linear Least Squares](https://mlsanity.com/paper/2303.03092/) (2023)
- [Optimization-based Causal Estimation from Heterogenous Environments](https://mlsanity.com/paper/2109.11990/) (2021)
- [Causal Invariance Learning via Efficient Nonconvex Optimization](https://mlsanity.com/paper/2412.11850/) (2024)
- [Fundamental Computational Limits in Pursuing Invariant Causal Prediction and Invariance-Guided Regularization](https://mlsanity.com/paper/2501.17354/) (2025)
- [Bayesian Hierarchical Invariant Prediction](https://mlsanity.com/paper/2505.11211/) (2025)

## Built on by (most cited first, 2 of 2)

- [Environment-Adaptive Covariate Selection: Learning When to Use Spurious Correlations for Out-of-Distribution Prediction](https://mlsanity.com/paper/2601.02322/) (2026)
- [Discrete Causal Representations from Heterogeneous Domains: A Bayesian Approach with Social Survey Applications](https://mlsanity.com/paper/2606.06288/) (2026)

## Abstract (arXiv)

Invariant prediction [Peters et al., 2016] analyzes feature/outcome data from multiple environments to identify invariant features - those with a stable predictive relationship to the outcome. Such features support generalization to new environments and help reveal causal mechanisms. Previous methods have primarily tackled this problem through hypothesis testing or regularized optimization. Here we develop Bayesian Invariant Prediction (BIP), a probabilistic model for invariant prediction. BIP encodes the indices of invariant features as a latent variable and recover them by posterior inference. Under the assumptions of Peters et al. [2016], the BIP posterior targets the true invariant features. We prove that the posterior is consistent and that greater environment heterogeneity leads to faster posterior contraction. To handle many features, we design an efficient variational approximation called VI-BIP. In simulations and real data, we find that BIP and VI-BIP are more accurate and scalable than existing methods for invariant prediction.