# Emergence of Linear Truth Encodings in Language Models

arXiv: https://arxiv.org/abs/2510.15804 · HTML: https://mlsanity.com/paper/2510.15804/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 34 references that are in the map.
- First posted on arXiv 17 Oct 2025.

## Builds on (most cited first, 30 of 31)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space](https://mlsanity.com/paper/2203.14680/) (2022)
- [LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations](https://mlsanity.com/paper/2410.02707/) (2024)
- [Birth of a Transformer: A Memory Viewpoint](https://mlsanity.com/paper/2306.00802/) (2023)
- [Analyzing Transformers in Embedding Space](https://mlsanity.com/paper/2209.02535/) (2022)
- [Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models](https://mlsanity.com/paper/2411.14257/) (2024)
- [Linear Adversarial Concept Erasure](https://mlsanity.com/paper/2201.12091/) (2022)
- [Eliciting Human Preferences with Language Models](https://mlsanity.com/paper/2310.11589/) (2023)
- [On the Origins of Linear Representations in Large Language Models](https://mlsanity.com/paper/2403.03867/) (2024)
- [Truth is Universal: Robust Detection of Lies in LLMs](https://mlsanity.com/paper/2407.12831/) (2024)
- [Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?](https://mlsanity.com/paper/2312.03729/) (2023)
- [Confidence Regulation Neurons in Language Models](https://mlsanity.com/paper/2406.16254/) (2024)
- [The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models](https://mlsanity.com/paper/2310.11877/) (2023)
- [Understanding Factual Recall in Transformers via Associative Memories](https://mlsanity.com/paper/2412.06538/) (2024)
- [Representation Surgery: Theory and Practice of Affine Steering](https://mlsanity.com/paper/2402.09631/) (2024)
- [Personas as a Way to Model Truthfulness in Language Models](https://mlsanity.com/paper/2310.18168/) (2023)
- [Who's asking? User personas and the mechanics of latent misalignment](https://mlsanity.com/paper/2406.12094/) (2024)
- [Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinations](https://mlsanity.com/paper/2403.18167/) (2024)
- [Scaling Laws for Associative Memories](https://mlsanity.com/paper/2310.02984/) (2023)
- [LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Models](https://mlsanity.com/paper/2204.12130/) (2022)
- [Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation](https://mlsanity.com/paper/2009.09435/) (2020)
- [Learning Associative Memories with Gradient Descent](https://mlsanity.com/paper/2402.18724/) (2024)

## Built on by (most cited first, 12 of 12)

- [Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering](https://mlsanity.com/paper/2511.00617/) (2025)
- [Can LLMs Introspect? A Reality Check](https://mlsanity.com/paper/2605.26242/) (2026)
- [Linear representations in language models can change dramatically over a conversation](https://mlsanity.com/paper/2601.20834/) (2026)
- [The Truthfulness Spectrum Hypothesis](https://mlsanity.com/paper/2602.20273/) (2026)
- [Hallucinations Undermine Trust; Metacognition is a Way Forward](https://mlsanity.com/paper/2605.01428/) (2026)
- [Why Do LLMs Struggle in Strategic Play? Broken Links Between Observations, Beliefs, and Actions](https://mlsanity.com/paper/2605.00226/) (2026)
- [Truth as a Compression Artifact in Language Model Training](https://mlsanity.com/paper/2603.11749/) (2026)
- [Geometric Factual Recall in Transformers](https://mlsanity.com/paper/2605.12426/) (2026)
- [Linear representations of grammaticality in neural language models](https://mlsanity.com/paper/2607.15175/) (2026)
- [Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness](https://mlsanity.com/paper/2609.03887/) (2026)
- [Linear Script Representations in Speech Foundation Models Enable Zero-Shot Transliteration](https://mlsanity.com/paper/2601.02906/) (2026)
- [What LLMs explain is not what they believe: Evaluating explanation sufficiency under models' own input beliefs](https://mlsanity.com/paper/2606.28615/) (2026)

## Abstract (arXiv)

Recent probing studies reveal that large language models exhibit linear subspaces that separate true from false statements, yet the mechanism behind their emergence is unclear. We introduce a transparent, one-layer transformer toy model that reproduces such truth subspaces end-to-end and exposes one concrete route by which they can arise. We study one simple setting in which truth encoding can emerge: a data distribution where factual statements co-occur with other factual statements (and vice-versa), encouraging the model to learn this distinction in order to lower the LM loss on future tokens. We corroborate this pattern with experiments in pretrained language models. Finally, in the toy setting we observe a two-phase learning dynamic: networks first memorize individual factual associations in a few steps, then -- over a longer horizon -- learn to linearly separate true from false, which in turn lowers language-modeling loss. Together, these results provide both a mechanistic demonstration and an empirical motivation for how and why linear truth representations can emerge in language models.