mlsanityData to 7 Oct 2026

The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Shared Category Geometry in Small Language Models

arXiv 2607.16741 ↗First posted 18 Jul 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

B"urger et al. (2024) demonstrated that truth representations in large language models are universal across statement polarity but reside within a multidimensional subspace. The truth value of a statement is linearly readable from a residual stream of language model, but it is not clear how much of that representation fits on a single direction, which component builds it, or what it is made of. We conducted a study based on these questions, with one instrument: a training-free axis, the dominant direction of the singular value decomposition (SVD) of hidden-state differences over true/false minimal pairs, identified without labels up to one global sign. Extensive evaluation across 14 models from 6 diverse architectural families (including MoE), read and extract at cost O(d) per token. We close with a pre-registered prediction on whether the arrangement extends to categories whose truth is computed rather than retrieved.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 14 of its 18 references that are in the map. First posted on arXiv 18 Jul 2026.

Builds on

14 of 18 references.

References · 14

  1. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  4. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  5. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  6. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  7. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  8. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  9. 2024The Platonic Representation Hypothesiscited by 362
  10. 2024Truth is Universal: Robust Detection of Lies in LLMscited by 78
  11. 2025ICR Probe: Tracking Hidden State Dynamics for Reliable Hallucination Detection in LLMscited by 31
  12. 2025Beyond Token Probes: Hallucination Detection via Activation Tensors with ACT-ViTcited by 16
2 more
  1. 2025LLM Hallucination Detection: A Fast Fourier Transform Method Based on Hidden Layer Temporal Signalscited by 4
  2. 2026How Transformers Reject Wrong Answers: Rotational Dynamics of Factual Constraint Processingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning
  2. 2025Large Language Models Encode Semantics and Alignment in Linearly Separable Representations
  3. 2026Dynamics Reveals Structure: Challenging the Linear Propagation Assumption
  4. 2026Beyond Language: Format-Agnostic Reasoning Subspaces in Large Language Models
  5. 2026Grounding Without Corrective Control: Truth-Tracking Profiles for Large Language Models
  6. 2026Confidence Geometry Reveals Trace-Level Correctness in Large Language Model Reasoning
  7. 2026Diagnosing Correctness Probes under Self-Judgement Confounding
  8. 2023Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds