# Identifying Representations for Intervention Extrapolation

arXiv: https://arxiv.org/abs/2310.04295 · HTML: https://mlsanity.com/paper/2310.04295/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 65 references that are in the map.
- First posted on arXiv 6 Oct 2023.

## Builds on (most cited first, 30 of 36)

- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Diffusion Autoencoders: Toward a Meaningful and Decodable Representation](https://mlsanity.com/paper/2111.15640/) (2021)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning](https://mlsanity.com/paper/1805.08651/) (2018)
- [Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA](https://mlsanity.com/paper/1605.06336/) (2016)
- [The Risks of Invariant Risk Minimization](https://mlsanity.com/paper/2010.05761/) (2020)
- [Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA](https://mlsanity.com/paper/2107.10098/) (2021)
- [Interventional Causal Representation Learning](https://mlsanity.com/paper/2209.11924/) (2022)
- [Weakly supervised causal representation learning](https://mlsanity.com/paper/2203.16437/) (2022)
- [Anchor regression: heterogeneous data meets causality](https://mlsanity.com/paper/1801.06229/) (2018)
- [Nonparametric Identifiability of Causal Representations from Unknown Interventions](https://mlsanity.com/paper/2306.00542/) (2023)
- [On Linear Identifiability of Learned Representations](https://mlsanity.com/paper/2007.00810/) (2020)
- [Learning Linear Causal Representations from Interventions under General Nonlinear Mixing](https://mlsanity.com/paper/2306.02235/) (2023)
- [Linear Causal Disentanglement via Interventions](https://mlsanity.com/paper/2211.16467/) (2022)
- [Identifiability Guarantees for Causal Disentanglement from Soft Interventions](https://mlsanity.com/paper/2307.06250/) (2023)
- [Identifiability of deep generative models without auxiliary information](https://mlsanity.com/paper/2206.10044/) (2022)
- [Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalization](https://mlsanity.com/paper/2202.06856/) (2022)
- [A causal framework for distribution generalization](https://mlsanity.com/paper/2006.07433/) (2020)
- [Weakly Supervised Representation Learning with Sparse Perturbations](https://mlsanity.com/paper/2206.01101/) (2022)
- [Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA](https://mlsanity.com/paper/2106.09620/) (2021)
- [Learning latent causal graphs via mixture oracles](https://mlsanity.com/paper/2106.15563/) (2021)
- [Identifiable Deep Generative Models via Sparse Decoding](https://mlsanity.com/paper/2110.10804/) (2021)
- [Score-based Causal Representation Learning with Interventions](https://mlsanity.com/paper/2301.08230/) (2023)
- [Learning nonparametric latent causal graphs with unknown interventions](https://mlsanity.com/paper/2306.02899/) (2023)
- [Identification of Nonlinear Latent Hierarchical Models](https://mlsanity.com/paper/2306.07916/) (2023)
- [Estimating the effect of joint interventions from observational data in sparse high-dimensional settings](https://mlsanity.com/paper/1407.2451/) (2014)
- [Kernel Conditional Moment Test via Maximum Moment Restriction](https://mlsanity.com/paper/2002.09225/) (2020)
- [Exploiting Independent Instruments: Identification and Distribution Generalization](https://mlsanity.com/paper/2202.01864/) (2022)

## Built on by (most cited first, 21 of 21)

- [Identifying Linearly-Mixed Causal Representations from Multi-Node Interventions](https://mlsanity.com/paper/2311.02695/) (2023)
- [Towards Interpretable Deep Generative Models via Causal Representation Learning](https://mlsanity.com/paper/2504.11609/) (2025)
- [Identifiability Guarantees for Causal Disentanglement from Purely Observational Data](https://mlsanity.com/paper/2410.23620/) (2024)
- [What is causal about causal models and representations?](https://mlsanity.com/paper/2501.19335/) (2025)
- [Invariance & Causal Representation Learning: Prospects and Limitations](https://mlsanity.com/paper/2312.03580/) (2023)
- [Extrapolation-Aware Nonparametric Statistical Inference](https://mlsanity.com/paper/2402.09758/) (2024)
- [Sanity Checking Causal Representation Learning on a Simple Real-World System](https://mlsanity.com/paper/2502.20099/) (2025)
- [Towards Understanding Extrapolation: a Causal Lens](https://mlsanity.com/paper/2501.09163/) (2025)
- [Causality-Inspired Robustness for Nonlinear Models via Representation Learning](https://mlsanity.com/paper/2505.12868/) (2025)
- [Structured Learning of Compositional Sequential Interventions](https://mlsanity.com/paper/2406.05745/) (2024)
- [Distributional Instrumental Variable Method](https://mlsanity.com/paper/2502.07641/) (2025)
- [Learning Causal Response Representations through Direct Effect Analysis](https://mlsanity.com/paper/2503.04358/) (2025)
- [SEDGE: Structural Extrapolated Data Generation](https://mlsanity.com/paper/2604.02482/) (2026)
- [Concept Modulation Models: A Unified Framework for Identifiability and Extrapolation](https://mlsanity.com/paper/2606.18509/) (2026)
- [Extrapolation Guarantees for Perturbation Modeling Under the Additive Latent Shift Assumption](https://mlsanity.com/paper/2504.18522/) (2025)
- [Learning Treatment Representations for Downstream Instrumental Variable Regression](https://mlsanity.com/paper/2506.02200/) (2025)
- [Incorporating Interventional Independence Improves Robustness against Interventional Distribution Shift](https://mlsanity.com/paper/2507.05412/) (2025)
- [Trek-Based Parameter Identification for Linear Causal Models With Arbitrarily Structured Latent Variables](https://mlsanity.com/paper/2507.18170/) (2025)
- [Anti-causal domain generalization: Leveraging unlabeled data](https://mlsanity.com/paper/2602.17187/) (2026)
- [Learning to Extrapolate to New Tasks: A Relational Approach to Task Extrapolation](https://mlsanity.com/paper/2605.30132/) (2026)
- [Causal Retention in Interactive Agents: Interface Factorization and Selective Adaptation](https://mlsanity.com/paper/2609.30650/) (2026)

## Abstract (arXiv)

The premise of identifiable and causal representation learning is to improve the current representation learning paradigm in terms of generalizability or robustness. Despite recent progress in questions of identifiability, more theoretical results demonstrating concrete advantages of these methods for downstream tasks are needed. In this paper, we consider the task of intervention extrapolation: predicting how interventions affect an outcome, even when those interventions are not observed at training time, and show that identifiable representations can provide an effective solution to this task even if the interventions affect the outcome non-linearly. Our setup includes an outcome Y, observed features X, which are generated as a non-linear transformation of latent features Z, and exogenous action variables A, which influence Z. The objective of intervention extrapolation is to predict how interventions on A that lie outside the training support of A affect Y. Here, extrapolation becomes possible if the effect of A on Z is linear and the residual when regressing Z on A has full support. As Z is latent, we combine the task of intervention extrapolation with identifiable representation learning, which we call Rep4Ex: we aim to map the observed features X into a subspace that allows for non-linear extrapolation in A. We show that the hidden representation is identifiable up to an affine transformation in Z-space, which is sufficient for intervention extrapolation. The identifiability is characterized by a novel constraint describing the linearity assumption of A on Z. Based on this insight, we propose a method that enforces the linear invariance constraint and can be combined with any type of autoencoder. We validate our theoretical findings through synthetic experiments and show that our approach succeeds in predicting the effects of unseen interventions.