mlsanityData to 14 Oct 2026

A Survey on the Linear Representation Hypothesis

arXiv 2609.22695 ↗First posted 19 Sep 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The term "linear representation hypothesis" (LRH) has appeared across diverse subfields of artificial intelligence, neuroscience, and cognitive science. But previous works have not consistently treated the LRH as a falsifiable scientific hypothesis; we analyze these inconsistencies and examine their implications for how prior theoretical and methodological results should be interpreted. Based on this analysis, we argue that claims regarding linear representations become well-defined only through careful examination of the model, representation location, feature definition, and evaluation dataset. We therefore propose a more rigorous formalization of the LRH that makes these dependencies explicit and allows the hypothesis to be evaluated as a falsifiable scientific claim. Finally, we identify some non-trivial open problems that warrant further attention from the research community.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 55 of its 93 references that are in the map. First posted on arXiv 19 Sep 2026.

Builds on

55 of 93 references.

References · 55

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  4. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  5. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  6. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  7. 2023Steering Language Models With Activation Engineeringcited by 922
  8. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  9. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 684
  10. 2019Designing and Interpreting Probes with Control Taskscited by 640
  11. 2021Causal Abstractions of Neural Networkscited by 422
  12. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
38 more
  1. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
  2. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 232
  3. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 212
  4. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 209
  5. 2020Information-Theoretic Probing for Linguistic Structurecited by 207
  6. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 200
  7. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 160
  8. 2015Order-Embeddings of Images and Languagecited by 113
  9. 2024Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Controlcited by 86
  10. 2024On the Origins of Linear Representations in Large Language Modelscited by 79
  11. 2025Language Models Use Trigonometry to Do Additioncited by 70
  12. 2018Probabilistic Embedding of Knowledge Graphs with Box Lattice Measurescited by 64
  13. 2020Pareto Probing: Trading Off Accuracy for Complexitycited by 55
  14. 2025JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulationcited by 54
  15. 2024The Geometry of Concepts: Sparse Autoencoder Feature Structurecited by 52
  16. 2024Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Modelscited by 51
  17. 2025From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuitcited by 49
  18. 2024Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representationscited by 47
  19. 2026Building Production-Ready Probes For Geminicited by 47
  20. 2022ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotationscited by 44
  21. 2025FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steeringcited by 40
  22. 2025The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?cited by 37
  23. 2021A Non-Linear Structural Probecited by 31
  24. 2024Cross-Entropy Is All You Need To Invert the Data Generating Processcited by 31
  25. 2024Refusal in LLMs is an Affine Functioncited by 29
  26. 2024Out-of-distribution generalization via composition: a lens through induction heads in Transformerscited by 28
  27. 2025Towards Understanding Distilled Reasoning Models: A Representational Approachcited by 26
  28. 2025On Linear Representations and Pretraining Data Frequency in Language Modelscited by 23
  29. 2025Transferring Linear Features Across Language Models With Model Stitchingcited by 21
  30. 2024A Reply to Makelov et al. (2023)'s "Interpretability Illusion" Argumentscited by 17
  31. 2024Towards a theory of model distillationcited by 15
  32. 2025I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?cited by 13
  33. 2024How Reliable are Causal Probing Interventions?cited by 11
  34. 2024Words in Motion: Extracting Interpretable Control Vectors for Motion Transformerscited by 9
  35. 2025Evaluating and Designing Sparse Autoencoders by Approximating Quasi-Orthogonalitycited by 8
  36. 2025Adversarial Attacks Leverage Interference Between Features in Superpositioncited by 8
  37. 2025Superpose Task-specific Features for Model Mergingcited by 5
  38. 2025Specialization after Generalization: Towards Understanding Test-Time Training in Foundation Modelscited by 5

50 most cited of 55 All 55 (JSON)

Built on by

Not cited yet.