The Truth Lies Somewhere in the Middle (of the Generated Tokens)
Not on the map. Why?
Abstract (arXiv)
How should hidden states generated autoregressively be collapsed into a representation that reflects a language model's internal state? Despite tokens being generated under causal masking, we find that mean pooling across their hidden states yields more semantic representations than any individual token alone. We quantify this through kernel alignment to reference spaces in language, vision, and protein domains. The improvement through mean pooling is consistent with information being distributed across generated tokens rather than localized to a single position. Furthermore, representations derived from generated tokens outperform those from prompt tokens, and alignment across generation reveals interpretable dynamics in model behavior.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 48 of its 61 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
48 of 61 references.
References · 48
- 2017Attention Is All You Needcited by 60,361
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2025Qwen3 Technical Reportcited by 8,820
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2023Let's Verify Step by Stepcited by 3,789
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2019Similarity of Neural Network Representations Revisitedcited by 1,755
- 2025DINOv3cited by 1,414
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2022Toy Models of Superpositioncited by 971
36 more
- 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
- 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
- 2023The Internal State of an LLM Knows When It's Lyingcited by 659
- 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
- 2024The Platonic Representation Hypothesiscited by 382
- 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 361
- 2023Language Models Represent Space and Timecited by 316
- 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
- 2022Masked Siamese Networks for Label-Efficient Learningcited by 302
- 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 287
- 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 245
- 2024Generative Representational Instruction Tuningcited by 233
- 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 159
- 2023Query2doc: Query Expansion with Large Language Modelscited by 149
- 2023Getting aligned on representational alignmentcited by 143
- 2025Why do LLMs attend to the first token?cited by 127
- 2023Future Lens: Anticipating Subsequent Tokens from a Single Hidden Statecited by 121
- 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 117
- 2023A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captionscited by 108
- 2024Not All Layers of LLMs Are Necessary During Inferencecited by 96
- 2022Reliability of CKA as a Similarity Measure in Deep Learningcited by 87
- 2023Scaling Sentence Embeddings with Large Language Modelscited by 85
- 2024Repetition Improves Language Model Embeddingscited by 72
- 2025Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoningcited by 51
- 2025Knowing Before Saying: LLM Representations Encode Information About Chain-of-Thought Success Before Completioncited by 23
- 2023Meaning Representations from Trajectories in Autoregressive Modelscited by 18
- 2024Correcting Biased Centered Kernel Alignment Measures in Biological and Artificial Neural Networkscited by 18
- 2024Token Prepending: A Training-Free Approach for Eliciting Better Sentence Embeddings from LLMscited by 18
- 2025Universally Converging Representations of Matter Across Scientific Foundation Modelscited by 14
- 2025Contrastive Prompting Enhances Sentence Embeddings in LLMs through Inference-Time Steeringcited by 13
- 2023Language Models are Universal Embedderscited by 11
- 2025Words That Make Language Models Perceivecited by 7
- 2025Dynamic Reflections: Probing Video Representations with Text Alignmentcited by 5
- 2024Aligning Large Language Models and Geometric Deep Models for Protein Representationcited by 3
- 2026Why Mean Pooling Works: Quantifying Second-Order Collapse in Text Embeddingscited by 2
Built on by
Citing papers per year: 2026: 3.
3 papers.