mlsanityData to 14 Oct 2026

The Spectral Geometry of Thought: Phase Transitions, Instruction Reversal, Token-Level Dynamics, and Perfect Correctness Prediction in How Transformers Reason

arXiv 2604.15350 ↗First posted 3 Apr 2026cs.LGcited by 1

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

We discover that large language models exhibit spectral phase transitions in their hidden activation spaces when engaging in reasoning versus factual recall. Through systematic spectral analysis across 11 models spanning 5 architecture families (Qwen, Pythia, Phi, Llama, DeepSeek-R1), we identify seven core phenomena: (1)~Reasoning Spectral Compression -- 9/11 models show significantly lower α for reasoning (p < 0.05), with larger effects in stronger models; (2)~Instruction Tuning Spectral Reversal -- base models show reasoning α < factual α, while instruction-tuned models reverse this relationship; (3)~Architecture-Dependent Generation Taxonomy -- prompt-to-response shifts partition into expansion, compression, and equilibrium regimes; (4)~Spectral Scaling Law -- αreasoning propto -0.074 ln N across 4 Qwen base models (R2 = 0.46); (5)~Token-Level Spectral Cascade -- per-token alpha tracking reveals local synchronization that decays exponentially with layer distance, and is weaker for reasoning than factual tasks; (6)~Reasoning Step Spectral Punctuation -- phase-transition signatures align with reasoning step boundaries; and (7)~Spectral Correctness Prediction -- spectral α alone achieves AUC = 1.000 (Qwen2.5-7B, late layers) and mean AUC = 0.893 across 6 models in predicting correctness before the final answer is generated. Together, these findings establish a comprehensive spectral theory of reasoning in transformers, revealing that the geometry of thought is universal in direction, architecture-specific in dynamics, and predictive of outcome.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 5 of its 37 references that are in the map. First posted on arXiv 3 Apr 2026.

Builds on

5 of 37 references.

References · 5

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  2. 2024Qwen2.5 Technical Reportcited by 4,942
  3. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  4. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  5. 2024Massive Activations in Large Language Modelscited by 294

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Geometry of Reason: Spectral Signatures of Valid Mathematical Reasoning
  2. 2026The Geometry of Thought: How Scale Restructures Reasoning In Large Language Models
  3. 2026Unveiling the Reasoning Process of Large Language Models
  4. 2026Reasoning emerges from constrained inference manifolds in large language models
  5. 2026Reasoning as State Transition: A Representational Analysis of Reasoning Evolution in Large Language Models
  6. 2026Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning
  7. 2025The Mirror Loop: Recursive Non-Convergence in Generative Reasoning Systems
  8. 2025Scaling Competence, Shrinking Reasoning: Cognitive Signatures in Language Model Learning