# Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semantics

arXiv: https://arxiv.org/abs/2403.01509 · HTML: https://mlsanity.com/paper/2403.01509/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 3 Mar 2024.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Causal Abstractions of Neural Networks](https://mlsanity.com/paper/2106.02997/) (2021)
- [WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations](https://mlsanity.com/paper/1808.09121/) (2018)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning](https://mlsanity.com/paper/2305.14160/) (2023)
- [Advances in Pre-Training Distributed Word Representations](https://mlsanity.com/paper/1712.09405/) (2017)
- [ChatGPT: Jack of all trades, master of none](https://mlsanity.com/paper/2302.10724/) (2023)
- [The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives](https://mlsanity.com/paper/1909.01380/) (2019)
- [Representation Of Lexical Stylistic Features In Language Models' Embedding Space](https://mlsanity.com/paper/2305.18657/) (2023)
- [LIAAD at SemDeep-5 Challenge: Word-in-Context (WiC)](https://mlsanity.com/paper/1906.10002/) (2019)
- [Scalar Adjective Identification and Multilingual Ranking](https://mlsanity.com/paper/2105.01180/) (2021)

## Built on by (most cited first, 30 of 44)

- [The Remarkable Robustness of LLMs: Stages of Inference?](https://mlsanity.com/paper/2406.19384/) (2024)
- [SEAL: Steerable Reasoning Calibration of Large Language Models for Free](https://mlsanity.com/paper/2504.07986/) (2025)
- [GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment](https://mlsanity.com/paper/2505.12215/) (2025)
- [Token Prepending: A Training-Free Approach for Eliciting Better Sentence Embeddings from LLMs](https://mlsanity.com/paper/2412.11556/) (2024)
- [Language Model Uncertainty Quantification with Attention Chain](https://mlsanity.com/paper/2503.19168/) (2025)
- [Contrastive Prompting Enhances Sentence Embeddings in LLMs through Inference-Time Steering](https://mlsanity.com/paper/2505.12831/) (2025)
- [A Comprehensive Study of Decoder-Only LLMs for Text-to-Image Generation](https://mlsanity.com/paper/2506.08210/) (2025)
- [Layerwise Recall and the Geometry of Interwoven Knowledge in LLMs](https://mlsanity.com/paper/2502.10871/) (2025)
- [Tracing Moral Foundations in Large Language Models](https://mlsanity.com/paper/2601.05437/) (2026)
- [Spectral Insights into Data-Oblivious Critical Layers in Large Language Models](https://mlsanity.com/paper/2506.00382/) (2025)
- [Semantic Routing: Exploring Multi-Layer LLM Feature Weighting for Diffusion Transformers](https://mlsanity.com/paper/2602.03510/) (2026)
- [NITP: Next Implicit Token Prediction for LLM Pre-training](https://mlsanity.com/paper/2605.24956/) (2026)
- [On-the-Fly Adaptive Distillation of Transformer to Dual-State Linear Attention](https://mlsanity.com/paper/2506.09316/) (2025)
- [LayerCake: Token-Aware Contrastive Decoding within Large Language Model Layers](https://mlsanity.com/paper/2507.04404/) (2025)
- [TRIM: Token-wise Attention-Derived Saliency for Data-Efficient Instruction Tuning](https://mlsanity.com/paper/2510.07118/) (2025)
- [Hierarchical Token Prepending: Enhancing Information Flow in Decoder-based LLM Embeddings](https://mlsanity.com/paper/2511.14868/) (2025)
- [From the New World of Word Embeddings: A Comparative Study of Small-World Lexico-Semantic Networks in LLMs](https://mlsanity.com/paper/2502.11380/) (2025)
- [qa-FLoRA: Data-free query-adaptive Fusion of LoRAs for LLMs](https://mlsanity.com/paper/2512.11366/) (2025)
- [Where meaning lives: Layer-wise accessibility of psycholinguistic features in encoder and decoder language models](https://mlsanity.com/paper/2601.03798/) (2026)
- [From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs](https://mlsanity.com/paper/2602.00628/) (2026)
- [Emergent effects of scaling on the functional hierarchies within large language models](https://mlsanity.com/paper/2501.07359/) (2025)
- [Beyond Prompting: An Efficient Embedding Framework for Open-Domain Question Answering](https://mlsanity.com/paper/2503.01606/) (2025)
- [Self-ReS: Self-Reflection in Large Vision-Language Models for Long Video Understanding](https://mlsanity.com/paper/2503.20362/) (2025)
- [Plug-in and Fine-tuning: Bridging the Gap between Small Language Models and Large Language Models](https://mlsanity.com/paper/2506.07424/) (2025)
- [Binary Autoencoder for Mechanistic Interpretability of Large Language Models](https://mlsanity.com/paper/2509.20997/) (2025)
- [Learning to Look at the Other Side: A Semantic Probing Study of Word Embeddings in LLMs with Enabled Bidirectional Attention](https://mlsanity.com/paper/2510.01652/) (2025)
- [PREGEN: Uncovering Latent Thoughts in Composed Video Retrieval](https://mlsanity.com/paper/2601.13797/) (2026)
- [Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Models](https://mlsanity.com/paper/2602.16608/) (2026)
- [Capsule Lens: Locating and Tracking Concept Geometry in Model Representations](https://mlsanity.com/paper/2609.05575/) (2026)
- [JuniperLiu at CoMeDi Shared Task: Models as Annotators in Lexical Semantics Disagreements](https://mlsanity.com/paper/2411.12147/) (2024)

## Abstract (arXiv)

Large language models have achieved remarkable success in general language understanding tasks. However, as a family of generative methods with the objective of next token prediction, the semantic evolution with the depth of these models are not fully explored, unlike their predecessors, such as BERT-like architectures. In this paper, we specifically investigate the bottom-up evolution of lexical semantics for a popular LLM, namely Llama2, by probing its hidden states at the end of each layer using a contextualized word identification task. Our experiments show that the representations in lower layers encode lexical semantics, while the higher layers, with weaker semantic induction, are responsible for prediction. This is in contrast to models with discriminative objectives, such as mask language modeling, where the higher layers obtain better lexical semantics. The conclusion is further supported by the monotonic increase in performance via the hidden states for the last meaningless symbols, such as punctuation, in the prompting strategy. Our codes are available at https://github.com/RyanLiut/LLM_LexSem.