# Extrapolation Guarantees for Perturbation Modeling Under the Additive Latent Shift Assumption

arXiv: https://arxiv.org/abs/2504.18522 · HTML: https://mlsanity.com/paper/2504.18522/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 37 of its 88 references that are in the map.
- First posted on arXiv 25 Apr 2025.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Inductive Biases for Deep Learning of Higher-Level Cognition](https://mlsanity.com/paper/2011.15091/) (2020)
- [Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA](https://mlsanity.com/paper/2107.10098/) (2021)
- [Foundations of Structural Causal Models with Cycles and Latent Variables](https://mlsanity.com/paper/1611.06221/) (2016)
- [Fader Networks: Manipulating Images by Sliding Attributes](https://mlsanity.com/paper/1706.00409/) (2017)
- [Nonparametric Identifiability of Causal Representations from Unknown Interventions](https://mlsanity.com/paper/2306.00542/) (2023)
- [Learning Linear Causal Representations from Interventions under General Nonlinear Mixing](https://mlsanity.com/paper/2306.02235/) (2023)
- [Linear Causal Disentanglement via Interventions](https://mlsanity.com/paper/2211.16467/) (2022)
- [Identifiability Guarantees for Causal Disentanglement from Soft Interventions](https://mlsanity.com/paper/2307.06250/) (2023)
- [Compositional Generalization from First Principles](https://mlsanity.com/paper/2307.05596/) (2023)
- [Visual Representation Learning Does Not Generalize Strongly Within the Same Domain](https://mlsanity.com/paper/2107.08221/) (2021)
- [Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation](https://mlsanity.com/paper/2307.02598/) (2023)
- [Engression: Extrapolation through the Lens of Distributional Regression](https://mlsanity.com/paper/2307.00835/) (2023)
- [Predicting Cellular Responses to Novel Drug Perturbations at a Single-Cell Resolution](https://mlsanity.com/paper/2204.13545/) (2022)
- [Provably Learning Object-Centric Representations](https://mlsanity.com/paper/2305.14229/) (2023)
- [Provable Compositional Generalization for Object-Centric Learning](https://mlsanity.com/paper/2310.05327/) (2023)
- [Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies](https://mlsanity.com/paper/2401.04890/) (2024)
- [Modelling Cellular Perturbations with the Sparse Additive Mechanism Shift Variational Autoencoder](https://mlsanity.com/paper/2311.02794/) (2023)
- [Learning Causal Representations of Single Cells via Sparse Mechanism Shift Modeling](https://mlsanity.com/paper/2211.03553/) (2022)
- [Score-based Causal Representation Learning: Linear and General Transformations](https://mlsanity.com/paper/2402.00849/) (2024)
- [Synthetic Interventions](https://mlsanity.com/paper/2006.07691/) (2020)
- [Distributional Diffusion Models with Scoring Rules](https://mlsanity.com/paper/2502.02483/) (2025)
- [First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains](https://mlsanity.com/paper/2211.11719/) (2022)
- [Identifying Representations for Intervention Extrapolation](https://mlsanity.com/paper/2310.04295/) (2023)
- [When does compositional structure yield compositional generalization? A kernel theory](https://mlsanity.com/paper/2405.16391/) (2024)
- [Interaction Asymmetry: A General Principle for Learning Composable Abstractions](https://mlsanity.com/paper/2411.07784/) (2024)
- [DISCO Nets: DISsimilarity COefficient Networks](https://mlsanity.com/paper/1606.02556/) (2016)
- [Causal Imputation via Synthetic Interventions](https://mlsanity.com/paper/2011.03127/) (2020)

## Abstract (arXiv)

We consider the problem of modeling the effects of perturbations like gene knockouts on measurements such as single-cell RNA counts. Given data for some perturbations, we aim to predict the distribution of measurements for new combinations of perturbations. To address this challenging extrapolation task, we posit that perturbations act additively in a suitable, unknown embedding space. We formulate the data-generating process as a latent variable model, in which perturbations amount to mean shifts in latent space and can be combined additively. We then prove that, given sufficiently diverse training perturbations, the representation and perturbation effects are identifiable up to orthogonal transformation and use this to derive extrapolation guarantees for unseen perturbations that can be expressed as linear combinations of seen ones. To estimate the model from data, we propose the perturbation distribution autoencoder (PDAE), which is trained by maximizing the distributional similarity between true and simulated perturbation distributions. The trained model can then be used to predict previously unseen perturbation distributions. In support of our theoretical results, we demonstrate through simulations that PDAE can accurately predict the effects of unseen but identifiable perturbations, and showcase the method on combinatorial gene perturbation data.