# The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Shared Category Geometry in Small Language Models

arXiv: https://arxiv.org/abs/2607.16741 · HTML: https://mlsanity.com/paper/2607.16741/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 18 references that are in the map.
- First posted on arXiv 18 Jul 2026.

## Builds on (most cited first, 14 of 14)

- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [The Platonic Representation Hypothesis](https://mlsanity.com/paper/2405.07987/) (2024)
- [Truth is Universal: Robust Detection of Lies in LLMs](https://mlsanity.com/paper/2407.12831/) (2024)
- [ICR Probe: Tracking Hidden State Dynamics for Reliable Hallucination Detection in LLMs](https://mlsanity.com/paper/2507.16488/) (2025)
- [Beyond Token Probes: Hallucination Detection via Activation Tensors with ACT-ViT](https://mlsanity.com/paper/2510.00296/) (2025)
- [LLM Hallucination Detection: A Fast Fourier Transform Method Based on Hidden Layer Temporal Signals](https://mlsanity.com/paper/2509.13154/) (2025)
- [How Transformers Reject Wrong Answers: Rotational Dynamics of Factual Constraint Processing](https://mlsanity.com/paper/2603.13259/) (2026)

## Abstract (arXiv)

B"urger et al. (2024) demonstrated that truth representations in large language models are universal across statement polarity but reside within a multidimensional subspace. The truth value of a statement is linearly readable from a residual stream of language model, but it is not clear how much of that representation fits on a single direction, which component builds it, or what it is made of. We conducted a study based on these questions, with one instrument: a training-free axis, the dominant direction of the singular value decomposition (SVD) of hidden-state differences over true/false minimal pairs, identified without labels up to one global sign. Extensive evaluation across 14 models from 6 diverse architectural families (including MoE), read and extract at cost O(d) per token. We close with a pre-registered prediction on whether the arrangement extends to categories whose truth is computed rather than retrieved.