# Out-of-distribution robustness for multivariate analysis via causal regularisation

arXiv: https://arxiv.org/abs/2403.01865 · HTML: https://mlsanity.com/paper/2403.01865/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 52 references that are in the map.
- First posted on arXiv 4 Mar 2024.

## Builds on (most cited first, 15 of 15)

- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations](https://mlsanity.com/paper/1505.05116/) (2015)
- [Anchor regression: heterogeneous data meets causality](https://mlsanity.com/paper/1801.06229/) (2018)
- [Recent Advances in Domain Adaptation for the Classification of Remote Sensing Data](https://mlsanity.com/paper/2104.07778/) (2021)
- [A causal framework for distribution generalization](https://mlsanity.com/paper/2006.07433/) (2020)
- [An Empirical Framework for Domain Generalization in Clinical Settings](https://mlsanity.com/paper/2103.11163/) (2021)
- [Regularizing towards Causal Invariance: Linear Models with Proxies](https://mlsanity.com/paper/2103.02477/) (2021)
- [Kernel Dependence Regularizers and Gaussian Processes with Applications to Algorithmic Fairness](https://mlsanity.com/paper/1911.04322/) (2019)
- [Distributional Anchor Regression](https://mlsanity.com/paper/2101.08224/) (2021)
- [Kernel Multivariate Analysis Framework for Supervised Subspace Learning: A Tutorial on Linear and Kernel Multivariate Methods](https://mlsanity.com/paper/1310.5089/) (2013)
- [Diagnosing the role of observable distribution shift in scientific replications](https://mlsanity.com/paper/2309.01056/) (2023)
- [Robust detection and attribution of climate change under interventions](https://mlsanity.com/paper/2212.04905/) (2022)

## Abstract (arXiv)

We propose a regularisation strategy of classical machine learning algorithms rooted in causality that ensures robustness against distribution shifts. Building upon the anchor regression framework, we demonstrate how incorporating a straightforward regularisation term into the loss function of classical multivariate analysis algorithms, such as (orthonormalized) partial least squares, reduced-rank regression, and multiple linear regression, enables out-of-distribution generalisation. Our framework allows users to efficiently verify the compatibility of a loss function with the regularisation strategy. Estimators for selected algorithms are provided, showcasing consistency and efficacy in synthetic and real-world climate science problems. The empirical validation highlights the versatility of anchor regularisation, emphasizing its compatibility with multivariate analysis approaches and its role in enhancing replicability while guarding against distribution shifts. The extended anchor framework advances causal inference methodologies, addressing the need for reliable out-of-distribution generalisation.