# Learning Causal Representations of Single Cells via Sparse Mechanism Shift Modeling

arXiv: https://arxiv.org/abs/2211.03553 · HTML: https://mlsanity.com/paper/2211.03553/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 71 references that are in the map.
- First posted on arXiv 7 Nov 2022.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA](https://mlsanity.com/paper/1605.06336/) (2016)
- [Causality for Machine Learning](https://mlsanity.com/paper/1911.10500/) (2019)
- [Differentiable Causal Discovery from Interventional Data](https://mlsanity.com/paper/2007.01754/) (2020)
- [Isolating Sources of Disentanglement in Variational Autoencoders](https://mlsanity.com/paper/1802.04942/) (2018)
- [Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA](https://mlsanity.com/paper/2107.10098/) (2021)
- [Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It](https://mlsanity.com/paper/1412.3730/) (2014)
- [Permutation-based Causal Inference Algorithms with Interventions](https://mlsanity.com/paper/1705.10220/) (2017)
- [Identifiable Deep Generative Models via Sparse Decoding](https://mlsanity.com/paper/2110.10804/) (2021)
- [Large-Scale Differentiable Causal Discovery of Factor Graphs](https://mlsanity.com/paper/2206.07824/) (2022)
- [Unsupervised Model Selection for Variational Disentangled Representation Learning](https://mlsanity.com/paper/1905.12614/) (2019)
- [The Case for Evaluating Causal Models Using Interventional Measures and Empirical Data](https://mlsanity.com/paper/1910.05387/) (2019)
- [Partial Disentanglement via Mechanism Sparsity](https://mlsanity.com/paper/2207.07732/) (2022)

## Built on by (most cited first, 30 of 34)

- [Modelling Cellular Perturbations with the Sparse Additive Mechanism Shift Variational Autoencoder](https://mlsanity.com/paper/2311.02794/) (2023)
- [PerturBench: Benchmarking Machine Learning Models for Cellular Perturbation Analysis](https://mlsanity.com/paper/2408.10609/) (2024)
- [Unpaired Multi-Domain Causal Representation Learning](https://mlsanity.com/paper/2302.00993/) (2023)
- [The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applications](https://mlsanity.com/paper/2412.01953/) (2024)
- [Learning Causal Representations from General Environments: Identifiability and Intrinsic Ambiguity](https://mlsanity.com/paper/2311.12267/) (2023)
- [Automated Discovery of Pairwise Interactions from Unstructured Data](https://mlsanity.com/paper/2409.07594/) (2024)
- [Invariance & Causal Representation Learning: Prospects and Limitations](https://mlsanity.com/paper/2312.03580/) (2023)
- [Sanity Checking Causal Representation Learning on a Simple Real-World System](https://mlsanity.com/paper/2502.20099/) (2025)
- [Out-of-Variable Generalization for Discriminative Models](https://mlsanity.com/paper/2304.07896/) (2023)
- [Toward the Identifiability of Comparative Deep Generative Models](https://mlsanity.com/paper/2401.15903/) (2024)
- [Learning Identifiable Factorized Causal Representations of Cellular Responses](https://mlsanity.com/paper/2410.22472/) (2024)
- [In-silico biological discovery with large perturbation models](https://mlsanity.com/paper/2503.23535/) (2025)
- [What Makes a Representation Good for Single-Cell Perturbation Prediction?](https://mlsanity.com/paper/2605.19343/) (2026)
- [Conditionally Invariant Representation Learning for Disentangling Cellular Heterogeneity](https://mlsanity.com/paper/2307.00558/) (2023)
- [Can sparse autoencoders make sense of gene expression latent variable models?](https://mlsanity.com/paper/2410.11468/) (2024)
- [Efficient Fine-Tuning of Single-Cell Foundation Models Enables Zero-Shot Molecular Perturbation Prediction](https://mlsanity.com/paper/2412.13478/) (2024)
- [No Foundations without Foundations -- Why semi-mechanistic models are essential for regulatory biology](https://mlsanity.com/paper/2501.19178/) (2025)
- [CRADLE-VAE: Enhancing Single-Cell Gene Perturbation Modeling with Counterfactual Reasoning-based Artifact Disentanglement](https://mlsanity.com/paper/2409.05484/) (2024)
- [Modeling Gene Expression Distributional Shifts for Unseen Genetic Perturbations](https://mlsanity.com/paper/2507.02980/) (2025)
- [sc-OTGM: Single-Cell Perturbation Modeling by Solving Optimal Mass Transport on the Manifold of Gaussian Mixtures](https://mlsanity.com/paper/2405.03726/) (2024)
- [Active learning for efficient discovery of optimal gene combinations in the combinatorial perturbation space](https://mlsanity.com/paper/2411.12010/) (2024)
- [Learning Latent Dynamical Causal Processes for Single-Cell Perturbation Prediction](https://mlsanity.com/paper/2605.25581/) (2026)
- [GPO-VAE: Modeling Explainable Gene Perturbation Responses utilizing GRN-Aligned Parameter Optimization](https://mlsanity.com/paper/2501.18973/) (2025)
- [Efficient Data Selection for Training Genomic Perturbation Models](https://mlsanity.com/paper/2503.14571/) (2025)
- [Extrapolation Guarantees for Perturbation Modeling Under the Additive Latent Shift Assumption](https://mlsanity.com/paper/2504.18522/) (2025)
- [Interpretable Causal Representation Learning for Biological Data in the Pathway Space](https://mlsanity.com/paper/2506.12439/) (2025)
- [Learning Robust Intervention Representations with Delta Embeddings](https://mlsanity.com/paper/2508.04492/) (2025)
- [Linear Causal Representation Learning by Topological Ordering, Pruning, and Disentanglement](https://mlsanity.com/paper/2509.22553/) (2025)
- [On the Identifiability of Causal Graphs with the Invariance Principle](https://mlsanity.com/paper/2510.13583/) (2025)
- [scCBGM: Interpretable Single-Cell Counterfactual Editing](https://mlsanity.com/paper/2606.07760/) (2026)

## Abstract (arXiv)

Latent variable models such as the Variational Auto-Encoder (VAE) have become a go-to tool for analyzing biological data, especially in the field of single-cell genomics. One remaining challenge is the interpretability of latent variables as biological processes that define a cell's identity. Outside of biological applications, this problem is commonly referred to as learning disentangled representations. Although several disentanglement-promoting variants of the VAE were introduced, and applied to single-cell genomics data, this task has been shown to be infeasible from independent and identically distributed measurements, without additional structure. Instead, recent methods propose to leverage non-stationary data, as well as the sparse mechanism shift assumption in order to learn disentangled representations with a causal semantic. Here, we extend the application of these methodological advances to the analysis of single-cell genomics data with genetic or chemical perturbations. More precisely, we propose a deep generative model of single-cell gene expression data for which each perturbation is treated as a stochastic intervention targeting an unknown, but sparse, subset of latent variables. We benchmark these methods on simulated single-cell data to evaluate their performance at latent units recovery, causal target identification and out-of-domain generalization. Finally, we apply those approaches to two real-world large-scale gene perturbation data sets and find that models that exploit the sparse mechanism shift hypothesis surpass contemporary methods on a transfer learning task. We implement our new model and benchmarks using the scvi-tools library, and release it as open-source software at https://github.com/Genentech/sVAE.