mlsanityData to 7 Oct 2026

Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models

arXiv 2402.09236 ↗First posted 14 Feb 2024cs.LG cs.AI math.STcited by 29

Not on the map. Why?

Abstract (arXiv)

To build intelligent machine learning systems, there are two broad approaches. One approach is to build inherently interpretable models, as endeavored by the growing field of causal representation learning. The other approach is to build highly-performant foundation models and then invest efforts into understanding how they work. In this work, we relate these two approaches and study how to learn human-interpretable concepts from data. Weaving together ideas from both fields, we formally define a notion of concepts and show that they can be provably recovered from diverse data. Experiments on synthetic data and large language models show the utility of our unified approach.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 83 of its 120 references that are in the map. First posted on arXiv 14 Feb 2024.

Builds on

83 of 120 references.

References · 83

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2013Intriguing properties of neural networkscited by 5,114
  9. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  11. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  12. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
38 more
  1. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  2. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  3. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  4. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  5. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  6. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  7. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  8. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  9. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  10. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  11. 2022Toy Models of Superpositioncited by 928
  12. 2023Steering Language Models With Activation Engineeringcited by 824
  13. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  14. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  15. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  16. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  17. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  18. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
  19. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  20. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  21. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  22. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  23. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  24. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  25. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
  26. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 278
  27. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 270
  28. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  29. 2016Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoderscited by 219
  30. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
  31. 2022Relative representations enable zero-shot latent space communicationcited by 175
  32. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 158
  33. 2023Linear Representations of Sentiment in Large Language Modelscited by 157
  34. 2017Learning to Generate Reviews and Discovering Sentimentcited by 147
  35. 2022Weakly supervised causal representation learningcited by 140
  36. 2022Interventional Causal Representation Learningcited by 140
  37. 2021Acquisition of Chess Knowledge in AlphaZerocited by 120
  38. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105

50 most cited of 83 All 83 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 12; 2026: 6.

29 papers.

Sort:

By area:

  1. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 130
  2. 2024On the Origins of Linear Representations in Large Language Modelscited by 76
  3. 2026Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behaviorcited by 22
  4. 2024On the Identification of Temporally Causal Representation with Instantaneous Dependencecited by 15
  5. 2024Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformerscited by 14
  6. 2024All or None: Identifiable Linear Properties of Next-token Predictors in Language Modelingcited by 14
  7. 2025I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?cited by 13
  8. 2025Towards Interpretable Deep Generative Models via Causal Representation Learningcited by 12
  9. 2025Incorporating Hierarchical Semantics in Sparse Autoencoder Architecturescited by 12
  10. 2024LogiCity: Advancing Neuro-Symbolic AI with Abstract Urban Simulationcited by 8
  11. 2024Identifiability Guarantees for Causal Disentanglement from Purely Observational Datacited by 7
  12. 2025CausalVLBench: Benchmarking Visual Causal Reasoning in Large Vision-Language Modelscited by 7
17 more
  1. 2025Learning General Causal Structures with Hidden Dynamic Process for Climate Analysiscited by 6
  2. 2024From Orthogonality to Dependency: Learning Disentangled Representation for Multi-Modal Time-Series Sensing Signalscited by 4
  3. 2024Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation modelscited by 4
  4. 2025Disentangling Long-Short Term State Under Unknown Interventions for Online Time Series Forecastingcited by 3
  5. 2026Causal Scaffolding for Physical Reasoning: A Benchmark for Causally-Informed Physical World Understanding in VLMscited by 3
  6. 2024Contrastive Sparse Autoencoders for Interpreting Planning of Chess-Playing Agentscited by 2
  7. 2025Causality Is Key to Understand and Balance Multiple Goals in Trustworthy ML and Foundation Modelscited by 2
  8. 2024A Theoretical Survey on Foundation Modelscited by 1
  9. 2025Time Series Domain Adaptation via Latent Invariant Causal Mechanismcited by 1
  10. 2025Causal View of Time Series Imputation: Some Identification Results on Missing Mechanismcited by 1
  11. 2025Intervening to Learn and Compose Causally Disentangled Representationscited by 1
  12. 2025Deciphering Personalization: Towards Fine-Grained Explainability in Natural Language for Personalized Image Generation Modelscited by 1
  13. 2026A Doubly Robust Machine Learning Approach for Disentangling Treatment Effect Heterogeneity with Functional Outcomescited by 1
  14. 2026Hierarchical Action Learning for Weakly-Supervised Action Segmentationcited by 1
  15. 2026Trustworthy AI Suffers from Invariance Conflicts and Causality is The Solutioncited by 1
  16. 2025Bringing Multi-Modal Multi-Task Federated Foundation Models to Education Domain: Prospects and Challengescited by 0
  17. 2026Concept Component Analysis: A Principled Approach for Concept Extraction in LLMscited by 0