mlsanityData to 7 Oct 2026

Extrapolation Guarantees for Perturbation Modeling Under the Additive Latent Shift Assumption

arXiv 2504.18522 ↗First posted 25 Apr 2025stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We consider the problem of modeling the effects of perturbations like gene knockouts on measurements such as single-cell RNA counts. Given data for some perturbations, we aim to predict the distribution of measurements for new combinations of perturbations. To address this challenging extrapolation task, we posit that perturbations act additively in a suitable, unknown embedding space. We formulate the data-generating process as a latent variable model, in which perturbations amount to mean shifts in latent space and can be combined additively. We then prove that, given sufficiently diverse training perturbations, the representation and perturbation effects are identifiable up to orthogonal transformation and use this to derive extrapolation guarantees for unseen perturbations that can be expressed as linear combinations of seen ones. To estimate the model from data, we propose the perturbation distribution autoencoder (PDAE), which is trained by maximizing the distributional similarity between true and simulated perturbation distributions. The trained model can then be used to predict previously unseen perturbation distributions. In support of our theoretical results, we demonstrate through simulations that PDAE can accurately predict the effects of unseen but identifiable perturbations, and showcase the method on combinatorial gene perturbation data.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 37 of its 88 references that are in the map. First posted on arXiv 25 Apr 2025.

Builds on

37 of 88 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018DAGs with NO TEARS: Continuous Optimization for Structure Learningcited by 745
  3. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
  4. 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 273
  5. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 158
  6. 2016Foundations of Structural Causal Models with Cycles and Latent Variablescited by 134
  7. 2017Fader Networks: Manipulating Images by Sliding Attributescited by 118
  8. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
  9. 2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
  10. 2022Linear Causal Disentanglement via Interventionscited by 84
  11. 2023Identifiability Guarantees for Causal Disentanglement from Soft Interventionscited by 84
  12. 2023Compositional Generalization from First Principlescited by 62
25 more
  1. 2021Visual Representation Learning Does Not Generalize Strongly Within the Same Domaincited by 60
  2. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 56
  3. 2023Engression: Extrapolation through the Lens of Distributional Regressioncited by 49
  4. 2023Provably Learning Object-Centric Representationscited by 45
  5. 2022Predicting Cellular Responses to Novel Drug Perturbations at a Single-Cell Resolutioncited by 42
  6. 2023Provable Compositional Generalization for Object-Centric Learningcited by 41
  7. 2024Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependenciescited by 39
  8. 2023Modelling Cellular Perturbations with the Sparse Additive Mechanism Shift Variational Autoencodercited by 35
  9. 2022Learning Causal Representations of Single Cells via Sparse Mechanism Shift Modelingcited by 33
  10. 2024Score-based Causal Representation Learning: Linear and General Transformationscited by 27
  11. 2020Synthetic Interventionscited by 25
  12. 2025Distributional Diffusion Models with Scoring Rulescited by 25
  13. 2023Identifying Representations for Intervention Extrapolationcited by 21
  14. 2022First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domainscited by 20
  15. 2024When does compositional structure yield compositional generalization? A kernel theorycited by 19
  16. 2024Interaction Asymmetry: A General Principle for Learning Composable Abstractionscited by 19
  17. 2016DISCO Nets: DISsimilarity COefficient Networkscited by 18
  18. 2020Causal Imputation via Synthetic Interventionscited by 13
  19. 2023Synthetic Combinations: A Causal Inference Framework for Combinatorial Interventionscited by 12
  20. 2024Season combinatorial intervention predictions with Salt & Pepercited by 11
  21. 2024Distributional Principal Autoencoderscited by 10
  22. 2024Automated Discovery of Pairwise Interactions from Unstructured Datacited by 8
  23. 2025Reverse Markov Learning: Multi-Step Generative Models for Complex Distributionscited by 7
  24. 2024Generative Intervention Models for Causal Perturbation Modelingcited by 4
  25. 2024General Causal Imputation via Synthetic Interventionscited by 1

Built on by

Not cited yet.