mlsanityData to 7 Oct 2026

Environment Invariant Linear Least Squares

arXiv 2303.03092 ↗First posted 6 Mar 2023math.ST cs.LG stat.MEcited by 17

Not on the map. Why?

Abstract (arXiv)

This paper considers a multi-environment linear regression model in which data from multiple experimental settings are collected. The joint distribution of the response variable and covariates may vary across different environments, yet the conditional expectations of y given the unknown set of important variables are invariant. Such a statistical model is related to the problem of endogeneity, causal inference, and transfer learning. The motivation behind it is illustrated by how the goals of prediction and attribution are inherent in estimating the true parameter and the important variable set. We construct a novel environment invariant linear least squares (EILLS) objective function, a multi-environment version of linear least-squares regression that leverages the above conditional expectation invariance structure and heterogeneity among different environments to determine the true parameter. Our proposed method is applicable without any additional structural knowledge and can identify the true parameter under a near-minimal identification condition. We establish non-asymptotic ℓ2 error bounds on the estimation error for the EILLS estimator in the presence of spurious variables. Moreover, we further show that the ℓ0 penalized EILLS estimator can achieve variable selection consistency in high-dimensional regimes. These non-asymptotic results demonstrate the sample efficiency of the EILLS estimator and its capability to circumvent the curse of endogeneity in an algorithmic manner without any prior structural knowledge. To the best of our knowledge, this paper is the first to realize statistically efficient invariance learning in the general linear model.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 16 of its 52 references that are in the map. First posted on arXiv 6 Mar 2023.

Builds on

16 of 52 references.

References · 16

  1. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  2. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  3. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  4. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
  5. 2020The Risks of Invariant Risk Minimizationcited by 262
  6. 2017Invariant Causal Prediction for Nonlinear Modelscited by 140
  7. 2018Anchor regression: heterogeneous data meets causalitycited by 116
  8. 2021Does Invariant Risk Minimization Capture Invariance?cited by 116
  9. 2020Invariant Causal Prediction for Block MDPscited by 104
  10. 2017Invariant Causal Prediction for Sequential Datacited by 63
  11. 2013Challenges of Big Data Analysiscited by 47
  12. 2021Nonlinear Invariant Risk Minimization: A Causal Approachcited by 42
4 more
  1. 2017Learning Causal Structures Using Regression Invariancecited by 32
  2. 2019Stabilizing Variable Selection and Regressioncited by 31
  3. 2021Optimization-based Causal Estimation from Heterogenous Environmentscited by 16
  4. 2017Causal Dantzig: fast inference in linear structural equation models with hidden variables under additive interventionscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 5; 2025: 6; 2026: 4.

17 papers.

Sort:

  1. 2024Minimax Regret Learning for Data with Heterogeneous Subgroupscited by 7
  2. 2024Causality Pursuit from Heterogeneous Environments via Neural Adversarial Invariance Learningcited by 7
  3. 2023Causality-oriented robustness: exploiting general noise interventionscited by 5
  4. 2025Achievable distributional robustness when the robust risk is only partially identifiedcited by 3
  5. 2023Multi-dimensional domain generalization with low-rank structurescited by 2
  6. 2024Causal Invariance Learning via Efficient Nonconvex Optimizationcited by 2
  7. 2025Bayesian Invariance Modeling of Multi-Environment Datacited by 2
  8. 2024The Implicit Bias of Heterogeneity towards Invariance: A Study of Multi-Environment Matrix Sensingcited by 1
  9. 2025DivIL: Unveiling and Addressing Over-Invariance for Out-of- Distribution Generalizationcited by 1
  10. 2025Bayesian Hierarchical Invariant Predictioncited by 1
  11. 2024FAIRM: Learning invariant representations for algorithmic fairness and domain generalization with minimax optimalitycited by 0
  12. 2025A Conditional Distribution Equality Testing Framework using Deep Generative Learningcited by 0
5 more
  1. 2025Invariant Feature Extraction Through Conditional Independence and the Optimal Transport Barycenter Problem: the Gaussian casecited by 0
  2. 2026Personalizing black-box models for nonparametric regression with minimax optimalitycited by 0
  3. 2026Environment-Adaptive Covariate Selection: Learning When to Use Spurious Correlations for Out-of-Distribution Predictioncited by 0
  4. 2026Beyond identifiability: Learning causal representations with few environments and finite samplescited by 0
  5. 2026Unveiling Invariant and Transferable Latent Factors Across Heterogeneous Environments via ATLAScited by 0