mlsanityData to 7 Oct 2026

Desiderata for Representation Learning: A Causal Perspective

arXiv 2109.03795 ↗First posted 8 Sep 2021stat.ML cs.LG stat.MEcited by 76

Not on the map. Why?

Abstract (arXiv)

Representation learning constructs low-dimensional representations to summarize essential features of high-dimensional data. This learning problem is often approached by describing various desiderata associated with learned representations; e.g., that they be non-spurious, efficient, or disentangled. It can be challenging, however, to turn these intuitive desiderata into formal criteria that can be measured and enhanced based on observed data. In this paper, we take a causal perspective on representation learning, formalizing non-spuriousness and efficiency (in supervised representation learning) and disentanglement (in unsupervised representation learning) using counterfactual quantities and observable consequences of causal assertions. This yields computable metrics that can be used to assess the degree to which representations satisfy the desiderata of interest and learn non-spurious and disentangled representations from single observational datasets.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). Builds on 49 of its 99 references that are in the map. First posted on arXiv 8 Sep 2021.

Builds on

49 of 99 references.

References · 49

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2019Invariant Risk Minimizationcited by 1,832
  5. 2016Variational Inference: A Review for Statisticianscited by 1,466
  6. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  7. 2018Disentangling by Factorisingcited by 614
  8. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
  9. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  10. 2020Environment Inference for Invariant Learningcited by 306
  11. 2017Variational Inference of Disentangled Latent Concepts from Unlabeled Observationscited by 240
  12. 2020Weakly-Supervised Disentanglement Without Compromisescited by 237
37 more
  1. 2017Emergence of Invariance and Disentanglement in Deep Representationscited by 213
  2. 2020Representation Learning via Invariant Causal Mechanismscited by 197
  3. 2018Isolating Sources of Disentanglement in Variational Autoencoderscited by 167
  4. 2018Invariant Representations without Adversarial Trainingcited by 111
  5. 2018Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustnesscited by 108
  6. 2017Learning Independent Causal Mechanismscited by 104
  7. 2018The Blessings of Multiple Causescited by 103
  8. 2017Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observationscited by 100
  9. 2013Probabilities of Causation: Bounds and Identificationcited by 89
  10. 2019Support and Invertibility in Domain-Invariant Representationscited by 87
  11. 2017Causal Structure Learningcited by 84
  12. 2019Weakly Supervised Disentanglement with Guaranteescited by 82
  13. 2021Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Testscited by 77
  14. 2020Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effectscited by 72
  15. 2020Robustness to Spurious Correlations in Text Classification via Automatically Generated Counterfactualscited by 67
  16. 2020Towards Unifying Feature Attribution and Counterfactual Explanations: Different Means to the Same Endcited by 66
  17. 2017Frequentist Consistency of Variational Bayescited by 60
  18. 2014Visual Causal Feature Learningcited by 59
  19. 2020Identifying Spurious Correlations for Robust Text Classificationcited by 59
  20. 2019Disentangling Factors of Variation Using Few Labelscited by 57
  21. 2021Explaining Black-Box Algorithms Using Probabilistic Contrastive Counterfactualscited by 55
  22. 2021Local Explanations via Necessity and Sufficiency: Unifying Theory and Practicecited by 45
  23. 2021Causes of Effects: Learning individual responses from population datacited by 36
  24. 2019Using Embeddings to Correct for Unobserved Confounding in Networkscited by 31
  25. 2014Stochastic Backpropagation and Approximate Inference in Deep Generative Modelscited by 30
  26. 2020Invariant Representation Learning for Treatment Effect Estimationcited by 25
  27. 2018Generalization in anti-causal learningcited by 23
  28. 2018Disentangling the independently controllable factors of variation by interacting with the worldcited by 16
  29. 2018Multiple Causal Inference with Latent Confoundingcited by 14
  30. 2017Independently Controllable Factorscited by 12
  31. 2020Towards Clarifying the Theory of the Deconfoundercited by 9
  32. 2021Causal Estimation with Functional Confounderscited by 9
  33. 2017Structured Latent Factor Analysis for Large-scale Data: Identifiability, Estimability, and Their Implicationscited by 8
  34. 2017Graph-based Isometry Invariant Representation Learningcited by 7
  35. 2019Disentangled Representation Learning with Wasserstein Total Correlationcited by 5
  36. 2015Causal and anti-causal learning in pattern recognition for neuroimagingcited by 4
  37. 2019Deep causal representation learning for unsupervised domain adaptationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 21; 2023: 18; 2024: 19; 2025: 10; 2026: 7.

76 papers.

Sort:

By area:

  1. 2022Interventional Causal Representation Learningcited by 140
  2. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
  3. 2023Causal Component Analysiscited by 56
  4. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 56
  5. 2021Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspectivecited by 40
  6. 2022Invariant and Transportable Representations for Anti-Causal Domain Shiftscited by 37
  7. 2022Disentanglement of Correlated Factors via Hausdorff Factorized Supportcited by 37
  8. 2022Indeterminacy in Generative Models: Characterization and Strong Identifiabilitycited by 33
  9. 2024The Essential Role of Causality in Foundation World Models for Embodied AIcited by 32
  10. 2024On the Challenges and Opportunities in Generative AIcited by 31
  11. 2022Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lenscited by 28
  12. 2023Counterfactual Learning on Graphs: A Surveycited by 22
38 more
  1. 2023Causal Reinforcement Learning: A Surveycited by 22
  2. 2023Invariant Learning via Probability of Sufficient and Necessary Causescited by 22
  3. 2023From Identifiable Causal Representations to Controllable Counterfactual Generation: A Survey on Causal Generative Modelingcited by 21
  4. 2023Towards Trustworthy Explanation: On Causal Rationalizationcited by 19
  5. 2022CPL: Counterfactual Prompt Learning for Vision and Language Modelscited by 18
  6. 2023Causal Context Connects Counterfactual Fairness to Robust Prediction and Group Fairnesscited by 17
  7. 2023Self-Supervised Disentanglement by Leveraging Structure in Data Augmentationscited by 16
  8. 2024Causal Inference with Generative Artificial Intelligence: Application to Texts as Treatmentscited by 16
  9. 2022Towards efficient representation identification in supervised learningcited by 14
  10. 2023On Learning Necessary and Sufficient Causal Graphscited by 14
  11. 2023Causal models in string diagramscited by 14
  12. 2024Recursive Causal Discoverycited by 12
  13. 2024Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environmentcited by 12
  14. 2025Towards Interpretable Deep Generative Models via Causal Representation Learningcited by 12
  15. 2022Towards a Grounded Theory of Causation for Embodied AIcited by 9
  16. 2022Data-Centric Epidemic Forecasting: A Surveycited by 8
  17. 2023A Measure-Theoretic Axiomatisation of Causalitycited by 8
  18. 2025Breaking Bad Tokens: Detoxification of LLMs Using Sparse Autoencoderscited by 8
  19. 2023C-Disentanglement: Discovering Causally-Independent Generative Factors under an Inductive Bias of Confoundercited by 7
  20. 2024Deep Autoregressive Models as Causal Inference Enginescited by 6
  21. 2025Compositional Causal Reasoning Evaluation in Language Modelscited by 6
  22. 2025Debiasing Reward Models by Representation Learning with Guaranteescited by 6
  23. 2022Learning Latent Structural Causal Modelscited by 5
  24. 2024FlexEControl: Flexible and Efficient Multimodal Control for Text-to-Image Generationcited by 5
  25. 2024Towards the Causal Complete Cause of Multi-Modal Representation Learningcited by 5
  26. 2022Generative multitask learning mitigates target-causing confoundingcited by 4
  27. 2022Sampling Through the Lens of Sequential Decision Makingcited by 4
  28. 2024CSRec: Rethinking Sequential Recommendation from A Causal Perspectivecited by 4
  29. 2024Identifying General Mechanism Shifts in Linear Causal Representationscited by 4
  30. 2025Discovering Hierarchical Latent Capabilities of Language Models via Causal Representation Learningcited by 4
  31. 2026Causal Representation Learning from General Environments under Nonparametric Mixingcited by 4
  32. 2022Improving Generalization via Uncertainty Driven Perturbationscited by 3
  33. 2022Towards Benchmarking Explainable Artificial Intelligence Methodscited by 3
  34. 2022Bias Challenges in Counterfactual Data Augmentationcited by 3
  35. 2022PaDPaF: Partial Disentanglement with Partially-Federated GANscited by 3
  36. 2023On the Identifiability of Quantized Factorscited by 3
  37. 2023Specify Robust Causal Representation from Mixed Observationscited by 3
  38. 2026CausalDetox: Causal Head Selection and Intervention for Language Model Detoxificationcited by 3

50 most cited of 76 All 76 (JSON)