# Causality Pursuit from Heterogeneous Environments via Neural Adversarial Invariance Learning

arXiv: https://arxiv.org/abs/2405.04715 · HTML: https://mlsanity.com/paper/2405.04715/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 89 references that are in the map.
- First posted on arXiv 7 May 2024.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Learning Models with Uniform Performance via Distributionally Robust Optimization](https://mlsanity.com/paper/1810.08750/) (2018)
- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks](https://mlsanity.com/paper/1703.02930/) (2017)
- [The Risks of Invariant Risk Minimization](https://mlsanity.com/paper/2010.05761/) (2020)
- [Causal Discovery with Continuous Additive Noise Models](https://mlsanity.com/paper/1309.6779/) (2013)
- [Deep Network Approximation for Smooth Functions](https://mlsanity.com/paper/2001.03040/) (2020)
- [Orthogonal Statistical Learning](https://mlsanity.com/paper/1901.09036/) (2019)
- [Invariant Causal Prediction for Nonlinear Models](https://mlsanity.com/paper/1706.08576/) (2017)
- [Anchor regression: heterogeneous data meets causality](https://mlsanity.com/paper/1801.06229/) (2018)
- [Does Invariant Risk Minimization Capture Invariance?](https://mlsanity.com/paper/2101.01134/) (2021)
- [Minimax Estimation of Conditional Moment Models](https://mlsanity.com/paper/2006.07201/) (2020)
- [How Well Generative Adversarial Networks Learn Distributions](https://mlsanity.com/paper/1811.03179/) (2018)
- [Invariant Causal Prediction for Sequential Data](https://mlsanity.com/paper/1706.08058/) (2017)
- [Strong Completeness and Faithfulness in Bayesian Networks](https://mlsanity.com/paper/1302.4973/) (2013)
- [Adversarial Estimation of Riesz Representers](https://mlsanity.com/paper/2101.00009/) (2020)
- [Augmented Minimax Linear Estimation](https://mlsanity.com/paper/1712.00038/) (2017)
- [Maximin effects in inhomogeneous large-scale data](https://mlsanity.com/paper/1406.0596/) (2014)
- [Learning Causal Structures Using Regression Invariance](https://mlsanity.com/paper/1705.09644/) (2017)
- [Stabilizing Variable Selection and Regression](https://mlsanity.com/paper/1911.01850/) (2019)
- [Minimax Regret Optimization for Robust Machine Learning under Distribution Shift](https://mlsanity.com/paper/2202.05436/) (2022)
- [Factor Augmented Sparse Throughput Deep ReLU Neural Networks for High Dimensional Regression](https://mlsanity.com/paper/2210.02002/) (2022)
- [Some Theoretical Properties of GANs](https://mlsanity.com/paper/1803.07819/) (2018)
- [Nonparametric Density Estimation under Adversarial Losses](https://mlsanity.com/paper/1805.08836/) (2018)
- [How do noise tails impact on deep ReLU networks?](https://mlsanity.com/paper/2203.10418/) (2022)
- [On the Logic of Causal Models](https://mlsanity.com/paper/1304.2355/) (2013)

## Built on by (most cited first, 7 of 7)

- [Optimally taming biases in black-box models for efficient semiparametric estimation](https://mlsanity.com/paper/2606.06368/) (2026)
- [CINDES: Classification induced neural density estimator and simulator](https://mlsanity.com/paper/2510.00367/) (2025)
- [Beyond identifiability: Learning causal representations with few environments and finite samples](https://mlsanity.com/paper/2603.25796/) (2026)
- [Optimal use of a black-box learner in semiparametric estimation](https://mlsanity.com/paper/2607.21541/) (2026)
- [Linear Causal Representation Learning by Topological Ordering, Pruning, and Disentanglement](https://mlsanity.com/paper/2509.22553/) (2025)
- [How Useful is Causal Invariance for Domain Adaptation in Finite-Sample Settings?](https://mlsanity.com/paper/2606.12680/) (2026)
- [Unveiling Invariant and Transferable Latent Factors Across Heterogeneous Environments via ATLAS](https://mlsanity.com/paper/2607.18209/) (2026)

## Abstract (arXiv)

Pursuing causality from data is a fundamental problem in scientific discovery, treatment intervention, and transfer learning. This paper introduces a novel algorithmic method for addressing nonparametric invariance and causality learning in regression models across multiple environments, where the joint distribution of response variables and covariates varies, but the conditional expectations of outcome given an unknown set of quasi-causal variables are invariant. The challenge of finding such an unknown set of quasi-causal or invariant variables is compounded by the presence of endogenous variables that have heterogeneous effects across different environments. The proposed Focused Adversarial Invariant Regularization (FAIR) framework utilizes an innovative minimax optimization approach that drives regression models toward prediction-invariant solutions through adversarial testing. Leveraging the representation power of neural networks, FAIR neural networks (FAIR-NN) are introduced for causality pursuit. It is shown that FAIR-NN can find the invariant variables and quasi-causal variables under a minimal identification condition and that the resulting procedure is adaptive to low-dimensional composition structures in a non-asymptotic analysis. Under a structural causal model, variables identified by FAIR-NN represent pragmatic causality and provably align with exact causal mechanisms under conditions of sufficient heterogeneity. Computationally, FAIR-NN employs a novel Gumbel approximation with decreased temperature and a stochastic gradient descent ascent algorithm. The procedures are demonstrated using simulated and real-data examples.