# Unifying Causal Representation Learning with the Invariance Principle

arXiv: https://arxiv.org/abs/2409.02772 · HTML: https://mlsanity.com/paper/2409.02772/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 75 of its 109 references that are in the map.
- First posted on arXiv 4 Sep 2024.

## Builds on (most cited first, 30 of 75)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [The Platonic Representation Hypothesis](https://mlsanity.com/paper/2405.07987/) (2024)
- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [Environment Inference for Invariant Learning](https://mlsanity.com/paper/2010.07249/) (2020)
- [Towards a Definition of Disentangled Representations](https://mlsanity.com/paper/1812.02230/) (2018)
- [Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning](https://mlsanity.com/paper/1805.08651/) (2018)
- [Causal Discovery with Continuous Additive Noise Models](https://mlsanity.com/paper/1309.6779/) (2013)
- [Weakly-Supervised Disentanglement Without Compromises](https://mlsanity.com/paper/2002.02886/) (2020)
- [Contrastive Learning Inverts the Data Generating Process](https://mlsanity.com/paper/2102.08850/) (2021)
- [Convergent Learning: Do different neural networks learn the same representations?](https://mlsanity.com/paper/1511.07543/) (2015)
- [Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.06607/) (2021)

## Built on by (most cited first, 30 of 31)

- [Towards Interpretable Deep Generative Models via Causal Representation Learning](https://mlsanity.com/paper/2504.11609/) (2025)
- [On the Value of Cross-Modal Misalignment in Multimodal Representation Learning](https://mlsanity.com/paper/2504.10143/) (2025)
- [Prediction-Powered Causal Inferences](https://mlsanity.com/paper/2502.06343/) (2025)
- [Position: An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Research](https://mlsanity.com/paper/2504.13101/) (2025)
- [Statistical and structural identifiability in representation learning](https://mlsanity.com/paper/2603.11970/) (2026)
- [Sanity Checking Causal Representation Learning on a Simple Real-World System](https://mlsanity.com/paper/2502.20099/) (2025)
- [The Third Pillar of Causal Analysis? A Measurement Perspective on Causal Representations](https://mlsanity.com/paper/2505.17708/) (2025)
- [When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspective](https://mlsanity.com/paper/2506.03784/) (2025)
- [Exploratory Causal Inference in SAEnce](https://mlsanity.com/paper/2510.14073/) (2025)
- [Distributional Equivalence in Linear Non-Gaussian Latent-Variable Cyclic Causal Models: Characterization and Learning](https://mlsanity.com/paper/2603.04780/) (2026)
- [Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attention](https://mlsanity.com/paper/2603.14483/) (2026)
- [Pre-trained Models Succeed in Medical Imaging with Representation Similarity Degradation](https://mlsanity.com/paper/2503.07958/) (2025)
- [Discovering What You Can Control: Interventional Boundary Discovery for Reinforcement Learning](https://mlsanity.com/paper/2603.18257/) (2026)
- [Causal Representation Learning with Observational Grouping for CXR Classification](https://mlsanity.com/paper/2506.20582/) (2025)
- [Intervening to Learn and Compose Causally Disentangled Representations](https://mlsanity.com/paper/2507.04754/) (2025)
- [Self-Supervised Learning from Structural Invariance](https://mlsanity.com/paper/2602.02381/) (2026)
- [Multi-Domain Empirical Bayes for Linearly-Mixed Causal Representations](https://mlsanity.com/paper/2603.18404/) (2026)
- [Concept Modulation Models: A Unified Framework for Identifiability and Extrapolation](https://mlsanity.com/paper/2606.18509/) (2026)
- [Learning Robust Intervention Representations with Delta Embeddings](https://mlsanity.com/paper/2508.04492/) (2025)
- [ROPES: Robotic Pose Estimation via Score-Based Causal Representation Learning](https://mlsanity.com/paper/2510.20884/) (2025)
- [Eigenfunction Extraction for Ordered Representation Learning](https://mlsanity.com/paper/2510.24672/) (2025)
- [Addressing Instrument-Outcome Confounding in Mendelian Randomization through Representation Learning](https://mlsanity.com/paper/2602.19782/) (2026)
- [Provable Subspace Identification of Nonlinear Multi-view CCA](https://mlsanity.com/paper/2602.23785/) (2026)
- [Causal Learning with the Invariance Principle](https://mlsanity.com/paper/2605.13589/) (2026)
- [A Dialogue between Causal and Traditional Representation Learning: Toward Mutual Benefits in a Unified Formulation](https://mlsanity.com/paper/2605.21058/) (2026)
- [The Perception-Physics Paradox: Probing Scientific Alignment with TC-Bench](https://mlsanity.com/paper/2605.24782/) (2026)
- [From Tokens to Policy: Causal and Interpretable Heterogeneous Treatment Effects Identification](https://mlsanity.com/paper/2606.17010/) (2026)
- [Disentangling Continuous-Time Latent Dynamics: Identifiability of Latent SDEs via Diffusion Shifts](https://mlsanity.com/paper/2606.28228/) (2026)
- [Concept Drift from a Causal Perspective](https://mlsanity.com/paper/2609.25340/) (2026)
- [λ-JEPA Spectral Anti-Collapse Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2609.35288/) (2026)

## Abstract (arXiv)

Causal representation learning (CRL) aims at recovering latent causal variables from high-dimensional observations to solve causal downstream tasks, such as predicting the effect of new interventions or more robust classification. A plethora of methods have been developed, each tackling carefully crafted problem settings that lead to different types of identifiability. These different settings are widely assumed to be important because they are often linked to different rungs of Pearl's causal hierarchy, even though this correspondence is not always exact. This work shows that instead of strictly conforming to this hierarchical mapping, many causal representation learning approaches methodologically align their representations with inherent data symmetries. Identification of causal variables is guided by invariance principles that are not necessarily causal. This result allows us to unify many existing approaches in a single method that can mix and match different assumptions, including non-causal ones, based on the invariance relevant to the problem at hand. It also significantly benefits applicability, which we demonstrate by improving treatment effect estimation on real-world high-dimensional ecological data. Overall, this paper clarifies the role of causal assumptions in the discovery of causal variables and shifts the focus to preserving data symmetries.