mlsanityData to 7 Oct 2026

Sanity Checking Causal Representation Learning on a Simple Real-World System

arXiv 2502.20099 ↗First posted 27 Feb 2025cs.LG cs.AI stat.MEcited by 5

Not on the map. Why?

Abstract (arXiv)

We evaluate methods for causal representation learning (CRL) on a simple, real-world system that satisfies the basic problem setup of CRL. The system consists of a controlled optical experiment producing a variety of measurements where the underlying causal factors---the control inputs to the experiment---are known, providing a ground truth. We select methods representative of different approaches to CRL and find that they all fail to consistently recover the underlying causal factors. To understand the failure modes of the evaluated algorithms, we perform an ablation on the data by substituting the real data-generating process with a simpler synthetic equivalent. The results reveal a reproducibility problem, as most methods already fail on this synthetic ablation despite its simple data-generating process. Additionally, we observe that common assumptions on the mixing function are crucial for the performance of some of the methods but do not hold in the real data. Our efforts highlight the contrast between the theoretical promise of the state of the art and the challenges in its application. We hope the benchmark serves as a simple, real-world sanity check to further develop and validate methodol- ogy, bridging the gap towards CRL methods that work in practice.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 38 of its 56 references that are in the map. First posted on arXiv 27 Feb 2025.

Builds on

38 of 56 references.

References · 38

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  4. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  5. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  6. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
  7. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
  8. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 270
  9. 2022Weakly supervised causal representation learningcited by 140
  10. 2022Interventional Causal Representation Learningcited by 140
  11. 2022CITRIS: Causal Identifiability from Temporal Intervened Sequencescited by 132
  12. 2020CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learningcited by 109
26 more
  1. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
  2. 2021Independent mechanism analysis, a new concept?cited by 96
  3. 2021Learning Temporally Causal Latent Processes from General Temporal Datacited by 96
  4. 2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
  5. 2019The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICAcited by 86
  6. 2022Linear Causal Disentanglement via Interventionscited by 84
  7. 2023Identifiability Guarantees for Causal Disentanglement from Soft Interventionscited by 84
  8. 2020Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effectscited by 72
  9. 2023Multi-View Causal Representation Learning with Partial Observabilitycited by 72
  10. 2023Identifiability Results for Multimodal Contrastive Learningcited by 58
  11. 2023Causal Component Analysiscited by 56
  12. 2022Causal Representation Learning for Instantaneous and Temporal Effects in Interactive Systemscited by 50
  13. 2023Score-based Causal Representation Learning with Interventionscited by 49
  14. 2022Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learningcited by 44
  15. 2024Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependenciescited by 39
  16. 2023BISCUIT: Causal Representation Learning from Binary Interactionscited by 38
  17. 2024A Sparsity Principle for Partially Observable Causal Representation Learningcited by 34
  18. 2022Learning Causal Representations of Single Cells via Sparse Mechanism Shift Modelingcited by 33
  19. 2024Unifying Causal Representation Learning with the Invariance Principlecited by 30
  20. 2022Partial Disentanglement via Mechanism Sparsitycited by 29
  21. 2023Multi-Domain Causal Representation Learning via Weak Distributional Invariancescited by 23
  22. 2023Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learningcited by 22
  23. 2024Marrying Causal Representation Learning with Dynamical Systems for Sciencecited by 22
  24. 2023Identifying Representations for Intervention Extrapolationcited by 21
  25. 2023Identifying Linearly-Mixed Causal Representations from Multi-Node Interventionscited by 18
  26. 2024Identifying General Mechanism Shifts in Linear Causal Representationscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025The Third Pillar of Causal Analysis? A Measurement Perspective on Causal Representationscited by 4
  2. 2026Structural Causal Bottleneck Modelscited by 4
  3. 2025CausalVerse: Benchmarking Causal Representation Learning with Configurable High-Fidelity Simulationscited by 3
  4. 2026Causal Retention in Interactive Agents: Interface Factorization and Selective Adaptationcited by 0
  5. 2026CIDER-FM: Foundation Models for Causal Inference from Diverse Experimental Regimescited by 0