mlsanityData to 7 Oct 2026

Towards Causal Representation Learning

arXiv 2102.11107 ↗First posted 22 Feb 2021cs.LG cs.AIcited by 207

Not on the map. Why?

Abstract (arXiv)

The two fields of machine learning and graphical causality arose and developed separately. However, there is now cross-pollination and increasing interest in both fields to benefit from the advances of the other. In the present paper, we review fundamental concepts of causal inference and relate them to crucial open problems of machine learning, including transfer and generalization, thereby assaying how causality can contribute to modern machine learning research. This also applies in the opposite direction: we note that most work in causality starts from the premise that the causal variables are given. A central problem for AI and causality is, thus, causal representation learning, the discovery of high-level causal variables from low-level observations. Finally, we delineate some implications of causality for machine learning and propose key research areas at the intersection of both communities.

Key facts. Cited by 207 papers in the map (data through 7 Oct 2026). Builds on 139 of its 279 references that are in the map. First posted on arXiv 22 Feb 2021.

Builds on

139 of 279 references.

References · 139

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  8. 2013Intriguing properties of neural networkscited by 5,114
  9. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  11. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  12. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
38 more
  1. 2019Invariant Risk Minimizationcited by 1,832
  2. 2018World Modelscited by 1,444
  3. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  4. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  8. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  9. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  10. 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
  11. 2020Object-Centric Learning with Slot Attentioncited by 865
  12. 2017Counterfactual Fairnesscited by 802
  13. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  14. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  15. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  16. 2019Solving Rubik's Cube with a Robot Handcited by 699
  17. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  18. 2018Recognition in Terra Incognitacited by 655
  19. 2018Disentangling by Factorisingcited by 614
  20. 2020Jukebox: A Generative Model for Musiccited by 535
  21. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
  22. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 503
  23. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
  24. 2019Making Convolutional Networks Shift-Invariant Againcited by 390
  25. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  26. 2018Detecting and Correcting for Label Shift with Black Box Predictorscited by 364
  27. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  28. 2017A simple neural network module for relational reasoningcited by 345
  29. 2018Neural Relational Inference for Interacting Systemscited by 339
  30. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 332
  31. 2013Causal Discovery with Continuous Additive Noise Modelscited by 252
  32. 2019Fast AutoAugmentcited by 243
  33. 2019Causality for Machine Learningcited by 242
  34. 2018Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shiftcited by 240
  35. 2020Weakly-Supervised Disentanglement Without Compromisescited by 237
  36. 2020On the Binding Problem in Artificial Neural Networkscited by 230
  37. 2019Recurrent Independent Mechanismscited by 219
  38. 2018Why do deep convolutional networks generalize so poorly to small image transformations?cited by 198

50 most cited of 139 All 139 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 31; 2022: 70; 2023: 41; 2024: 22; 2025: 22; 2026: 21.

207 papers.

Sort:

By area:

  1. 2021Domain Generalization: A Surveycited by 760
  2. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 603
  3. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
  4. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 219
  5. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 158
  6. 2022Weakly supervised causal representation learningcited by 140
  7. 2022Causality Inspired Representation Learning for Domain Generalizationcited by 136
  8. 2022CITRIS: Causal Identifiability from Temporal Intervened Sequencescited by 132
  9. 2025LLM Social Simulations Are a Promising Research Methodcited by 97
  10. 2023GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphscited by 96
  11. 2021Towards Principled Disentanglement for Domain Generalizationcited by 91
  12. 2023Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Taskcited by 91
38 more
  1. 2021Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalizationcited by 83
  2. 2021Efficient Neural Causal Discovery without Acyclicity Constraintscited by 82
  3. 2023Modular Deep Learningcited by 82
  4. 2021Generalization and Robustness Implications in Object-Centric Learningcited by 81
  5. 2021Causal Influence Detection for Improving Efficiency in Reinforcement Learningcited by 70
  6. 2022Causal Machine Learning for Healthcare and Precision Medicinecited by 68
  7. 2024Causal Representation Learning from Multiple Distributions: A General Settingcited by 67
  8. 2021Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modelingcited by 59
  9. 2022Transferability in Deep Learning: A Surveycited by 57
  10. 2022Assaying Out-Of-Distribution Generalization in Transfer Learningcited by 53
  11. 2023High Fidelity Image Counterfactuals with Probabilistic Causal Modelscited by 53
  12. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 52
  13. 2022Causal Representation Learning for Instantaneous and Temporal Effects in Interactive Systemscited by 50
  14. 2022Mechanistic Mode Connectivitycited by 50
  15. 2021AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learningcited by 46
  16. 2023Provably Learning Object-Centric Representationscited by 45
  17. 2022Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learningcited by 44
  18. 2022Causal de Finetti: On the Identification of Invariant Causal Structure in Exchangeable Datacited by 42
  19. 2021Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspectivecited by 40
  20. 2022Robust and Efficient Medical Imaging with Self-Supervisioncited by 40
  21. 2022Invariant and Transportable Representations for Anti-Causal Domain Shiftscited by 37
  22. 2023Modelling Cellular Perturbations with the Sparse Additive Mechanism Shift Variational Autoencodercited by 35
  23. 2021CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Modelscited by 33
  24. 2022Indeterminacy in Generative Models: Characterization and Strong Identifiabilitycited by 33
  25. 2024Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Spacecited by 33
  26. 2021On the Complexity of SHAP-Score-Based Explanations: Tractability via Knowledge Compilation and Non-Approximability Resultscited by 32
  27. 2023Leveraging sparse and shared feature activations for disentangled representation learningcited by 32
  28. 2022Invariant Causal Mechanisms through Distribution Matchingcited by 31
  29. 2021Properties from Mechanisms: An Equivariance Perspective on Identifiable Representation Learningcited by 30
  30. 2022Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQAcited by 30
  31. 2023Causal Representation Learning Made Identifiable by Grouping of Observational Variablescited by 30
  32. 2022Partial Disentanglement via Mechanism Sparsitycited by 29
  33. 2023Robust Generalization against Photon-Limited Corruptions via Worst-Case Sharpness Minimizationcited by 29
  34. 2024Unbiased Faster R-CNN for Single-source Domain Generalized Object Detectioncited by 29
  35. 2022Causally Testing Gender Bias in LLMs: A Case Study on Occupational Biascited by 28
  36. 2021Dynamic Inference with Neural Interpreterscited by 24
  37. 2022Explainable AI for clinical and remote health applications: a survey on tabular and time series datacited by 24
  38. 2022Evaluation Methods and Measures for Causal Learning Algorithmscited by 23

50 most cited of 207 All 207 (JSON)