mlsanityData to 14 Oct 2026

Selecting The Most Informative Tokens in Natural Language Autoencoders

arXiv 2609.37040 ↗First posted 29 Sep 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Natural language autoencoders translate a language model's internal activations into readable explanations. Explaining every token position is costly. Which positions should an auditor inspect to understand a potential threat? We study this question across 4.7 million explanations on prompt injection and concealment. We compare signals from model computation with a ranker trained only on chat structure. Chat structure usually selects more relevant explanations than the computational signals, without requiring a model forward pass for position selection. On three of four datasets, explaining just 5% of positions retains nearly all of the success rate from explaining every position, where success means obtaining an explanation about the threat. The benefit varies with the audit task. We also show that pretrained verbalizers recover words that models have learned to conceal through fine-tuning, without additional verbalizer training. These results identify where auditors can concentrate explanation generation and show that useful explanations can extend beyond the model a verbalizer was trained to describe.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 43 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

29 of 43 references.

References · 29

  1. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  4. 2025Gemma 3 Technical Reportcited by 1,824
  5. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  6. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  7. 2023Steering Language Models With Activation Engineeringcited by 922
  8. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  9. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  10. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  11. 2020Quantifying Attention Flow in Transformerscited by 775
  12. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
17 more
  1. 2024Massive Activations in Large Language Modelscited by 294
  2. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  3. 2023Language Modeling Is Compressioncited by 225
  4. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 162
  5. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Gamecited by 145
  6. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 142
  7. 2024Attention Tracker: Detecting Prompt Injection Attacks in LLMscited by 95
  8. 2024SelfIE: Self-Interpretation of Large Language Model Embeddingscited by 67
  9. 2024Compression Represents Intelligence Linearlycited by 56
  10. 2025Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainerscited by 52
  11. 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 51
  12. 2026The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinkscited by 27
  13. 2025Liars' Bench: Evaluating Lie Detectors for Language Modelscited by 22
  14. 2024Position: Explain to Question not to Justifycited by 20
  15. 2025Towards eliciting latent knowledge from LLMs with mechanistic interpretabilitycited by 19
  16. 2026Prompt Injection as Role Confusioncited by 17
  17. 2026Confidence and Calibration of Activation Oracles for Reliable Interpretation of Language Model Internalscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Eliciting Secret Knowledge from Language Models
  2. 2026When Do Explanations Help In-Context Learning? A Comparative Study of Natural Language Explanation Types and Faithfulness
  3. 2025Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations
  4. 2026SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalization
  5. 2026How Much Does a Reasoning Summary Reveal? An Observability Ladder for Large Language Models
  6. 2026Can Large Language Models Still Explain Themselves? Investigating the Impact of Quantization on Self-Explanations
  7. 2024When Backdoors Speak: Understanding LLM Backdoor Attacks Through Model-Generated Explanations
  8. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning