# Invariant Subspace Decomposition

arXiv: https://arxiv.org/abs/2404.09962 · HTML: https://mlsanity.com/paper/2404.09962/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 15 Apr 2024.

## Builds on (most cited first, 10 of 10)

- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Correlation Alignment for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1612.01939/) (2016)
- [Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions](https://mlsanity.com/paper/1707.06422/) (2017)
- [Invariant Causal Prediction for Sequential Data](https://mlsanity.com/paper/1706.08058/) (2017)
- [Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices](https://mlsanity.com/paper/1912.10754/) (2019)
- [Maximin effects in inhomogeneous large-scale data](https://mlsanity.com/paper/1406.0596/) (2014)
- [Stabilizing Variable Selection and Regression](https://mlsanity.com/paper/1911.01850/) (2019)
- [Magging: maximin aggregation for inhomogeneous large-scale data](https://mlsanity.com/paper/1409.2638/) (2014)
- [Robustifying Independent Component Analysis by Adjusting for Group-Wise Stationary Noise](https://mlsanity.com/paper/1806.01094/) (2018)
- [Invariant Models for Causal Transfer Learning](https://mlsanity.com/paper/1507.05333/) (2015)

## Built on by (most cited first, 2 of 2)

- [Anchor PCA](https://mlsanity.com/paper/2606.06233/) (2026)
- [Invariance-Based Dynamic Regret Minimization](https://mlsanity.com/paper/2603.03843/) (2026)

## Abstract (arXiv)

We consider the task of predicting a response Y from a set of covariates X in settings where the conditional distribution of Y given X changes over time. For this to be feasible, assumptions on how the conditional distribution changes over time are required. Existing approaches assume, for example, that changes occur smoothly over time so that short-term prediction using only the recent past becomes feasible. To additionally exploit observations further in the past, we propose a novel invariance-based framework for linear conditionals, called Invariant Subspace Decomposition (ISD), that splits the conditional distribution into a time-invariant and a residual time-dependent component. As we show, this decomposition can be utilized both for zero-shot and time-adaptation prediction tasks, that is, settings where either no or a small amount of training data is available at the time points we want to predict Y at, respectively. We propose a practical estimation procedure, which automatically infers the decomposition using tools from approximate joint matrix diagonalization. Furthermore, we provide finite sample guarantees for the proposed estimator and demonstrate empirically that it indeed improves on approaches that do not use the additional invariant structure.