mlsanityData to 7 Oct 2026

Anchor regression: heterogeneous data meets causality

arXiv 1801.06229 ↗First posted 18 Jan 2018stat.MEcited by 116

Not on the map. Why?

Abstract (arXiv)

We consider the problem of predicting a response variable from a set of covariates on a data set that differs in distribution from the training data. Causal parameters are optimal in terms of predictive accuracy if in the new distribution either many variables are affected by interventions or only some variables are affected, but the perturbations are strong. If the training and test distributions differ by a shift, causal parameters might be too conservative to perform well on the above task. This motivates anchor regression, a method that makes use of exogeneous variables to solve a relaxation of the causal minimax problem by considering a modification of the least-squares loss. The procedure naturally provides an interpolation between the solutions of ordinary least squares and two-stage least squares. We prove that the estimator satisfies predictive guarantees in terms of distributional robustness against shifts in a linear class; these guarantees are valid even if the instrumental variables assumptions are violated. If anchor regression and least squares provide the same answer (anchor stability), we establish that OLS parameters are invariant under certain distributional changes. Anchor regression is shown empirically to improve replicability and protect against distributional shifts.

Key facts. Cited by 116 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Jan 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 24; 2023: 26; 2024: 18; 2025: 13; 2026: 17.

116 papers.

Sort:

By area:

  1. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
  2. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  3. 2021On Calibration and Out-of-domain Generalizationcited by 134
  4. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
  5. 2021Independent mechanism analysis, a new concept?cited by 96
  6. 2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
  7. 2022Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalizationcited by 69
  8. 2022Probable Domain Generalization via Quantile Risk Minimizationcited by 66
  9. 2022Out-of-distribution Generalization with Causal Invariant Transformationscited by 45
  10. 2023Model Agnostic Sample Reweighting for Out-of-Distribution Learningcited by 43
  11. 2022Invariant and Transportable Representations for Anti-Causal Domain Shiftscited by 37
  12. 2023Diagnosing Model Performance Under Distribution Shiftcited by 37
38 more
  1. 2021An Empirical Framework for Domain Generalization in Clinical Settingscited by 32
  2. 2022Invariant Causal Mechanisms through Distribution Matchingcited by 31
  3. 2024Domain Generalization through Meta-Learning: A Surveycited by 28
  4. 2021Transport-based Counterfactual Modelscited by 27
  5. 2021Federated Causal Inference in Heterogeneous Observational Datacited by 24
  6. 2022Learning Invariant Visual Representations for Compositional Zero-Shot Learningcited by 24
  7. 2023Invariant Learning via Probability of Sufficient and Necessary Causescited by 22
  8. 2023Spuriosity Didn't Kill the Classifier: Using Invariant Predictions to Harness Spurious Featurescited by 21
  9. 2023Identifying Representations for Intervention Extrapolationcited by 21
  10. 2021Regularizing towards Causal Invariance: Linear Models with Proxiescited by 20
  11. 2022Exploiting Independent Instruments: Identification and Distribution Generalizationcited by 18
  12. 2021Invariant Policy Learning: A Causal Perspectivecited by 17
  13. 2023Environment Invariant Linear Least Squarescited by 17
  14. 2021Optimization-based Causal Estimation from Heterogenous Environmentscited by 16
  15. 2021Training Data Subset Selection for Regression with Controlled Generalization Errorcited by 14
  16. 2024Domain Generalisation via Imprecise Learningcited by 14
  17. 2022Characterization and Greedy Learning of Gaussian Structural Causal Models under Unknown Interventionscited by 12
  18. 2024The Causal Chambers: Real Physical Systems as a Testbed for AI Methodologycited by 12
  19. 2024Do causal predictors generalize better to new domains?cited by 11
  20. 2022Evaluating Robustness to Dataset Shift via Parametric Robustness Setscited by 10
  21. 2022Identifying Causal Effects using Instrumental Time Series: Nuisance IV and Correcting for the Pastcited by 9
  22. 2021Causal Effect Variational Autoencoder with Uniform Treatmentcited by 8
  23. 2022Functional Generalized Empirical Likelihood Estimation for Conditional Moment Restrictionscited by 8
  24. 2025Partial Transportability for Domain Generalizationcited by 8
  25. 2021Instrumental Variable-Driven Domain Generalization with Unobserved Confounderscited by 7
  26. 2024Causality Pursuit from Heterogeneous Environments via Neural Adversarial Invariance Learningcited by 7
  27. 2021Robust Generalization despite Distribution Shift via Minimum Discriminating Informationcited by 6
  28. 2023Learning under random distributional shiftscited by 6
  29. 2023Continuous Invariance Learningcited by 6
  30. 2024Proxy Methods for Domain Adaptationcited by 6
  31. 2024First-Order Manifold Data Augmentation for Regression Learningcited by 6
  32. 2024Progression: an extrapolation principle for regressioncited by 6
  33. 2022Learning Instrumental Variable from Data Fusion for Treatment Effect Estimationcited by 5
  34. 2023Causality-oriented robustness: exploiting general noise interventionscited by 5
  35. 2023Structural restrictions in local causal discovery: identifying direct causes of a target variablecited by 5
  36. 2023Prominent Roles of Conditionally Invariant Components in Domain Adaptation: Theory and Algorithmscited by 5
  37. 2023Anchor Data Augmentationcited by 5
  38. 2023Invariance & Causal Representation Learning: Prospects and Limitationscited by 5

50 most cited of 116 All 116 (JSON)