mlsanityData to 7 Oct 2026

Does Representation Matter? Exploring Intermediate Layers in Large Language Models

arXiv 2412.09563 ↗First posted 12 Dec 2024cs.LG cs.CLcited by 31

Not on the map. Why?

Abstract (arXiv)

Understanding what defines a good representation in large language models (LLMs) is fundamental to both theoretical understanding and practical applications. In this paper, we investigate the quality of intermediate representations in various LLM architectures, including Transformers and State Space Models (SSMs). We find that intermediate layers often yield more informative representations for downstream tasks than the final layers. To measure the representation quality, we adapt and apply a suite of metrics - such as prompt entropy, curvature, and augmentation-invariance - originally proposed in other contexts. Our empirical study reveals significant architectural differences, how representations evolve throughout training, and how factors like input randomness and prompt length affect each layer. Notably, we observe a bimodal pattern in the entropy of some intermediate layers and consider potential explanations tied to training data. Overall, our results illuminate the internal mechanics of LLMs and guide strategies for architectural optimization and training.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 31 of its 46 references that are in the map. First posted on arXiv 12 Dec 2024.

Builds on

31 of 46 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  6. 2016Pointer Sentinel Mixture Modelscited by 2,992
  7. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  8. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  9. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  10. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  11. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  12. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
19 more
  1. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  2. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 494
  3. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
  4. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 382
  5. 2022RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rankcited by 145
  6. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 138
  7. 2024Not All Layers of LLMs Are Necessary During Inferencecited by 95
  8. 2024BM25S: Orders of magnitude faster lexical search via eager sparse scoringcited by 87
  9. 2023To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Reviewcited by 72
  10. 2023Eliciting Latent Knowledge from Quirky Language Modelscited by 56
  11. 2020Emergence of Separable Manifolds in Deep Language Representationscited by 39
  12. 2022Information Theory with Kernel Methodscited by 37
  13. 2023Reverse Engineering Self-Supervised Learningcited by 33
  14. 2023Large language models implicitly learn to straighten neural sentence trajectories to construct a predictive representation of natural languagecited by 32
  15. 2023LiDAR: Sensing Linear Probing Performance in Joint Embedding SSL Architecturescited by 27
  16. 2022Guillotine Regularization: Why removing layers is needed to improve generalization in Self-Supervised Learningcited by 25
  17. 2022Information Flow in Deep Neural Networkscited by 22
  18. 2023DiME: Maximizing Mutual Information by a Difference of Matrix-Based Entropiescited by 20
  19. 2021Understanding Neural Networks with Logarithm Determinant Entropy Estimatorcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 20; 2026: 10.

31 papers.

Sort:

By area:

  1. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
  2. 2025A Survey on Latent Reasoningcited by 71
  3. 2025Defending against Indirect Prompt Injection by Instruction Detectioncited by 25
  4. 2025BiggerGait: Unlocking Gait Recognition with Layer-wise Representations from Large Vision Modelscited by 18
  5. 2025The Intrinsic Dimension of Prompts in Internal Representations of Large Language Modelscited by 17
  6. 2025Advancing LLM Safe Alignment with Safety Representation Rankingcited by 15
  7. 2025ARWKV: Pretrain is not what we need, an RNN-Attention-Based Language Model Born from Transformercited by 9
  8. 2025HiProbe-VAD: Video Anomaly Detection via Hidden States Probing in Tuning-Free Multimodal LLMscited by 8
  9. 2025Entropy-Lens: Uncovering Decision Strategies in LLMscited by 6
  10. 2025Efficient numeracy in language models through single-token number embeddingscited by 6
  11. 2024Lightweight Safety Classification Using Pruned Language Modelscited by 5
  12. 2026Improving Visual Reasoning with Iterative Evidence Refinementcited by 5
19 more
  1. 2025Explanations of Large Language Models Explain Language Representations in the Braincited by 4
  2. 2025Where Did It Go Wrong? Attributing Undesirable LLM Behaviors via Representation Gradient Tracingcited by 4
  3. 2026Latent Structure of Affective Representations in Large Language Modelscited by 3
  4. 2025CHAIR -- Classifier of Hallucination as Improvercited by 2
  5. 2025Large Language Models Show Signs of Alignment with Human Neurocognition During Abstract Reasoningcited by 2
  6. 2025Language Ranker: A Lightweight Ranking framework for LLM Decodingcited by 2
  7. 2025GRAN-TED: Generating Robust, Aligned, and Nuanced Text Embedding for Diffusion Modelscited by 2
  8. 2025LLaMAX2: Your Translation-Enhanced Model also Performs Well in Reasoningcited by 1
  9. 2025Concept Attractors in LLMs and their Applicationscited by 1
  10. 2026Geometry of Human Perceptual Domains Emerges Transiently in LLM Representationscited by 1
  11. 2025Blind Spot Navigation: Evolutionary Discovery of Sensitive Semantic Concepts for LVLMscited by 0
  12. 2025Intra-request branch orchestration for efficient LLM reasoningcited by 0
  13. 2026Language-Pretraining-Induced Bias: A Strong Foundation for General Vision Taskscited by 0
  14. 2026Multilingual Embedding Probes Fail to Generalize Across Learner Corporacited by 0
  15. 2026Towards Understanding the Robustness of Sparse Autoencoderscited by 0
  16. 2026Geometric and Information Compression of Representations in Deep Learningcited by 0
  17. 2026Gradient Smoothing: Coupling Layer-wise Updates for Improved Optimizationcited by 0
  18. 2026PermuFormer: Multi-Task Pretraining for Permutation Representation in Algebraic Combinatoricscited by 0
  19. 2026The Residual Stream's Effective Depthcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Layer by Layer: Uncovering Hidden Representations in Language Models
  2. 2025Demystifying the Roles of LLM Layers in Retrieval, Knowledge, and Reasoning
  3. 2026Global Divergence, Local Convergence: Representation Geometry in SSMs and Transformers
  4. 2025A Comparative Analysis of Contextual Representation Flow in State-Space and Transformer Architectures
  5. 2025How Language Directions Align with Token Geometry in Multilingual LLMs
  6. 2025SR-LLM: Rethinking the Structured Representation in Large Language Model
  7. 2025LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws
  8. 2025From Compression to Expression: A Layerwise Analysis of In-Context Learning