# Environment Invariant Linear Least Squares

arXiv: https://arxiv.org/abs/2303.03092 · HTML: https://mlsanity.com/paper/2303.03092/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 52 references that are in the map.
- First posted on arXiv 6 Mar 2023.

## Builds on (most cited first, 16 of 16)

- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [The Risks of Invariant Risk Minimization](https://mlsanity.com/paper/2010.05761/) (2020)
- [Invariant Causal Prediction for Nonlinear Models](https://mlsanity.com/paper/1706.08576/) (2017)
- [Anchor regression: heterogeneous data meets causality](https://mlsanity.com/paper/1801.06229/) (2018)
- [Does Invariant Risk Minimization Capture Invariance?](https://mlsanity.com/paper/2101.01134/) (2021)
- [Invariant Causal Prediction for Block MDPs](https://mlsanity.com/paper/2003.06016/) (2020)
- [Invariant Causal Prediction for Sequential Data](https://mlsanity.com/paper/1706.08058/) (2017)
- [Challenges of Big Data Analysis](https://mlsanity.com/paper/1308.1479/) (2013)
- [Nonlinear Invariant Risk Minimization: A Causal Approach](https://mlsanity.com/paper/2102.12353/) (2021)
- [Learning Causal Structures Using Regression Invariance](https://mlsanity.com/paper/1705.09644/) (2017)
- [Stabilizing Variable Selection and Regression](https://mlsanity.com/paper/1911.01850/) (2019)
- [Optimization-based Causal Estimation from Heterogenous Environments](https://mlsanity.com/paper/2109.11990/) (2021)
- [Causal Dantzig: fast inference in linear structural equation models with hidden variables under additive interventions](https://mlsanity.com/paper/1706.06159/) (2017)

## Built on by (most cited first, 17 of 17)

- [Minimax Regret Learning for Data with Heterogeneous Subgroups](https://mlsanity.com/paper/2405.01709/) (2024)
- [Causality Pursuit from Heterogeneous Environments via Neural Adversarial Invariance Learning](https://mlsanity.com/paper/2405.04715/) (2024)
- [Causality-oriented robustness: exploiting general noise interventions](https://mlsanity.com/paper/2307.10299/) (2023)
- [Achievable distributional robustness when the robust risk is only partially identified](https://mlsanity.com/paper/2502.02710/) (2025)
- [Multi-dimensional domain generalization with low-rank structures](https://mlsanity.com/paper/2309.09555/) (2023)
- [The Implicit Bias of Heterogeneity towards Invariance: A Study of Multi-Environment Matrix Sensing](https://mlsanity.com/paper/2403.01420/) (2024)
- [Causal Invariance Learning via Efficient Nonconvex Optimization](https://mlsanity.com/paper/2412.11850/) (2024)
- [Bayesian Invariance Modeling of Multi-Environment Data](https://mlsanity.com/paper/2506.22675/) (2025)
- [DivIL: Unveiling and Addressing Over-Invariance for Out-of- Distribution Generalization](https://mlsanity.com/paper/2502.12413/) (2025)
- [Bayesian Hierarchical Invariant Prediction](https://mlsanity.com/paper/2505.11211/) (2025)
- [A Conditional Distribution Equality Testing Framework using Deep Generative Learning](https://mlsanity.com/paper/2509.17729/) (2025)
- [Environment-Adaptive Covariate Selection: Learning When to Use Spurious Correlations for Out-of-Distribution Prediction](https://mlsanity.com/paper/2601.02322/) (2026)
- [Beyond identifiability: Learning causal representations with few environments and finite samples](https://mlsanity.com/paper/2603.25796/) (2026)
- [FAIRM: Learning invariant representations for algorithmic fairness and domain generalization with minimax optimality](https://mlsanity.com/paper/2404.01608/) (2024)
- [Invariant Feature Extraction Through Conditional Independence and the Optimal Transport Barycenter Problem: the Gaussian case](https://mlsanity.com/paper/2512.20914/) (2025)
- [Personalizing black-box models for nonparametric regression with minimax optimality](https://mlsanity.com/paper/2601.01432/) (2026)
- [Unveiling Invariant and Transferable Latent Factors Across Heterogeneous Environments via ATLAS](https://mlsanity.com/paper/2607.18209/) (2026)

## Abstract (arXiv)

This paper considers a multi-environment linear regression model in which data from multiple experimental settings are collected. The joint distribution of the response variable and covariates may vary across different environments, yet the conditional expectations of y given the unknown set of important variables are invariant. Such a statistical model is related to the problem of endogeneity, causal inference, and transfer learning. The motivation behind it is illustrated by how the goals of prediction and attribution are inherent in estimating the true parameter and the important variable set. We construct a novel environment invariant linear least squares (EILLS) objective function, a multi-environment version of linear least-squares regression that leverages the above conditional expectation invariance structure and heterogeneity among different environments to determine the true parameter. Our proposed method is applicable without any additional structural knowledge and can identify the true parameter under a near-minimal identification condition. We establish non-asymptotic ℓ2 error bounds on the estimation error for the EILLS estimator in the presence of spurious variables. Moreover, we further show that the ℓ0 penalized EILLS estimator can achieve variable selection consistency in high-dimensional regimes. These non-asymptotic results demonstrate the sample efficiency of the EILLS estimator and its capability to circumvent the curse of endogeneity in an algorithmic manner without any prior structural knowledge. To the best of our knowledge, this paper is the first to realize statistically efficient invariance learning in the general linear model.