mlsanityData to 7 Oct 2026

From Isolation to Entanglement: When Do Interpretability Methods Identify and Disentangle Known Concepts?

arXiv 2512.15134 ↗First posted 17 Dec 2025cs.LG cs.AI cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

A goal of interpretability is to recover disentangled representations of latent concepts (features) from the activations of neural networks. The quality of features is typically evaluated in isolation, and under implicit independence assumptions that may not hold in practice. Thus, it is unclear to what extent common featurization methods such as sparse autoencoders (SAEs) and probes disentangle one concept from another. We propose a multi-concept evaluation setting using concepts including sentiment, domain, voice, and tense. We evaluate how well featurizers produce disentangled representations of each concept, observing that features are typically sensitive to only one concept, but also that concepts are distributed across many features. Then, we steer these features, measuring whether each concept is independently manipulable, and whether features interact. Even in idealized settings, steering a feature often affects many concepts, despite a near absence of interaction effects. These results suggest that correlational metrics are insufficient to establish steering selectivity, and that demonstrating that two features operate in separate spaces is insufficient to claim that they will be selective for one concept. These results underscore the importance of multi-concept evaluations in interpretability research.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 62 of its 84 references that are in the map. First posted on arXiv 17 Dec 2025.

Builds on

62 of 84 references.

References · 62

  1. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  2. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  5. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  6. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  7. 2024Scaling and evaluating sparse autoencoderscited by 530
  8. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
  9. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  10. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  11. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  12. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
38 more
  1. 2018Towards a Definition of Disentangled Representationscited by 294
  2. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 278
  3. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 270
  4. 2020Weakly-Supervised Disentanglement Without Compromisescited by 237
  5. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
  6. 2022Neural Networks and the Chomsky Hierarchycited by 214
  7. 2021Towards Causal Representation Learningcited by 207
  8. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 201
  9. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  10. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
  11. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 158
  12. 2018A Variational Inequality Perspective on Generative Adversarial Networkscited by 144
  13. 2022Weakly supervised causal representation learningcited by 140
  14. 2021Investigating the Limitations of Transformers with Simple Arithmetic Taskscited by 136
  15. 2022CITRIS: Causal Identifiability from Temporal Intervened Sequencescited by 132
  16. 2018Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustnesscited by 108
  17. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
  18. 2020ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICAcited by 102
  19. 2021Independent mechanism analysis, a new concept?cited by 96
  20. 2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
  21. 2019The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICAcited by 86
  22. 2023Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examplescited by 84
  23. 2023Identifiability Guarantees for Causal Disentanglement from Soft Interventionscited by 84
  24. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 77
  25. 2025SAEs Are Good for Steering -- If You Select the Right Featurescited by 74
  26. 2022Weakly Supervised Representation Learning with Sparse Perturbationscited by 64
  27. 2023Compositional Generalization from First Principlescited by 62
  28. 2025Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometrycited by 58
  29. 2019Disentangling Factors of Variation Using Few Labelscited by 57
  30. 2023Causal Component Analysiscited by 56
  31. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 56
  32. 2021Identifiable Deep Generative Models via Sparse Decodingcited by 54
  33. 2025MIB: A Mechanistic Interpretability Benchmarkcited by 49
  34. 2025From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuitcited by 45
  35. 2022Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learningcited by 44
  36. 2023Disentanglement via Latent Quantizationcited by 40
  37. 2023BISCUIT: Causal Representation Learning from Binary Interactionscited by 38
  38. 2022Embrace the Gap: VAEs Perform Independent Mechanism Analysiscited by 28

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Causality is Key for Interpretability Claims to Generalisecited by 8
  2. 2026Who Guards the Guardians? The Challenges of Evaluating Identifiability of Learned Representationscited by 0
  3. 2026Phase-Associative Memory: Sequence Modeling in Complex Hilbert Spacecited by 0
  4. 2026Tensor Product Representation Probes Reveal Shared Structure Across Linear Directionscited by 0
  5. 2026Disentanglement Beyond Generative Models with Riemannian ICAcited by 0