mlsanityData to 14 Oct 2026

Knowledge and representations inside LLMs

136 papers0 this weekMedium confidence in this grouping: mostly one topic, with blurry edgesMedium confidence in this grouping: mostly one topic, with blurry edges · About

Keywords: language, interpretability, llms, large, large language

Key facts. Knowledge and representations inside LLMs: 136 papers on arXiv since 2021 (data through 14 Oct 2026). 22 of its 136 mapped papers (16%) cite “Eliciting Latent Predictions from Transformers with the Tuned Lens” (2023, arXiv 2303.08112). 7 new papers 24 Sep–14 Oct.

What’s new

24 Sep – 14 Oct 2026

7 new papers, 24 Sep – 14 Oct; 0 this week.

  1. Earlier · 7
  2. SemanticFold: Latent Sequence Compression SeparatesLanguage Modeling, Decodability, and Reasoning also fits: SAE features and their applications
  3. Copying Before Suppression: What Drives a Below-Chance Dip During Language Model Training? also fits: Probing deception, bias and beliefs
  4. Clinical Concept Centers in LLMs
  5. Signatures of semantic search in the activations of large language models
  6. Spontaneous Context Restoration: How Language Models Recover from Corrupted Inputs also fits: Probing deception, bias and beliefs
  7. Identifying Temporal Features within Transcoders for Time Sensitive Factual Recall
  8. FTB Graph: Determining and Validating First-token Broadcasters and Language-Identity Head Circuits in Multilingual Language Models also fits: Circuits and mechanistic analysis

Foundations

Most cited by papers here

  1. Eliciting Latent Predictions from Transformers with the Tuned Lens2023cited by 22 here
  2. Towards Best Practices of Activation Patching in Language Models: Metrics and Methods2023cited by 16 here
  3. Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models2024cited by 14 here
  4. Representation Engineering: A Top-Down Approach to AI Transparency2023cited by 12 here
  5. Do Multilingual LLMs Think In English?2025cited by 3 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 45 here
  2. 2023Progress measures for grokking via mechanistic interpretabilitycited by 16 here
  3. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 28 here
  4. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 39 here
  5. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 16 here
  6. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 14 here
  7. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 17 here
  8. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 6 here
16 more it builds on, not in this chain
  1. 2022Toy Models of Superpositioncited by 16 here
  2. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 6 here
  3. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 22 here
  4. 2023Steering Language Models With Activation Engineeringcited by 9 here
  5. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 12 here
  6. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 9 here
  7. 2023Steering Llama 2 via Contrastive Activation Additioncited by 10 here
  8. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 9 here
  9. 2024Scaling and evaluating sparse autoencoderscited by 10 here
  10. 2024Transcoders Find Interpretable LLM Feature Circuitscited by 12 here
  11. 2024Who's asking? User personas and the mechanics of latent misalignmentcited by 5 here
  12. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 4 here
  13. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 12 here
  14. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 4 here
  15. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 5 here
  16. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 5 here

The 24 most cited of 196.

Browse

30 of 136, ranked 6–35

  1. 6Towards eliciting latent knowledge from LLMs with mechanistic interpretability20252
  2. 7Rethinking Interpretability in the Era of Large Language Models20242
  3. 8Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Models20242
  4. 9Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers20242
  5. 10Aligning Large Language Models with Human Preferences through Representation Engineering20232
  6. 11Towards Intrinsic Interpretability of Large Language Models:A Survey of Design Principles and Architectures20261
  7. 12RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs20251
  8. 13LogitLens4LLMs: Extending Logit Lens Analysis to Modern Large Language Models20251
  9. 14How do Large Language Models Understand Relevance? A Mechanistic Interpretability Perspective20251
  10. 15On Entity Identification in Language Models20251
All 30
  1. 16TRACE: Training and Inference-Time Interpretability Analysis for Language Models20251
  2. 17A Review of Developmental Interpretability in Large Language Models20251
  3. 18Beyond Transcription: Mechanistic Interpretability in ASR20251
  4. 19Interpreting Language Models Through Concept Descriptions: A Survey20251
  5. 20Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations20251
  6. 21Stream: Scaling up Mechanistic Interpretability to Long Context in LLMs via Sparse Attention20251
  7. 22Tracing Multilingual Representations in LLMs with Cross-Layer Transcoders20251
  8. 23Towards Uncovering How Large Language Model Works: An Explainability Perspective20241
  9. 24RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations20241
  10. 25Unlocking the Future: Exploring Look-Ahead Planning Mechanistic Interpretability in Large Language Models20241
  11. 26On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMs20241
  12. 27You can remove GPT2's LayerNorm by fine-tuning20241
  13. 28The Same But Different: Structural Similarities and Differences in Multilingual Language Modeling20241
  14. 29Character-level Chinese Backpack Language Models20231
  15. 30Proto-lm: A Prototypical Network-Based Framework for Built-in Interpretability in Large Language Models20231
  16. 31Semantic Gravity Wells: Why Negative Constraints Backfire20260
  17. 32From Interpretability to Performance: Optimizing Retrieval Heads for Long-Context Language Models20260
  18. 33Conversational Context Classification: A Representation Engineering Approach20260
  19. 34A Shared Geometry of Difficulty in Multilingual Language Models20260
  20. 35Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models20260