# The Spectral Geometry of Thought: Phase Transitions, Instruction Reversal, Token-Level Dynamics, and Perfect Correctness Prediction in How Transformers Reason

arXiv: https://arxiv.org/abs/2604.15350 · HTML: https://mlsanity.com/paper/2604.15350/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 5 of its 37 references that are in the map.
- First posted on arXiv 3 Apr 2026.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 5 of 5)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [Massive Activations in Large Language Models](https://mlsanity.com/paper/2402.17762/) (2024)

## Built on by (most cited first, 1 of 1)

- [KV-PRM: Efficient Process Reward Modeling via KV-Cache Transfer for Multi-Agent Test-Time Scaling](https://mlsanity.com/paper/2607.09153/) (2026)

## Abstract (arXiv)

We discover that large language models exhibit spectral phase transitions in their hidden activation spaces when engaging in reasoning versus factual recall. Through systematic spectral analysis across 11 models spanning 5 architecture families (Qwen, Pythia, Phi, Llama, DeepSeek-R1), we identify seven core phenomena: (1)~Reasoning Spectral Compression -- 9/11 models show significantly lower α for reasoning (p < 0.05), with larger effects in stronger models; (2)~Instruction Tuning Spectral Reversal -- base models show reasoning α <  factual α, while instruction-tuned models reverse this relationship; (3)~Architecture-Dependent Generation Taxonomy -- prompt-to-response shifts partition into expansion, compression, and equilibrium regimes; (4)~Spectral Scaling Law -- αreasoning propto -0.074 ln N across 4 Qwen base models (R2 = 0.46); (5)~Token-Level Spectral Cascade -- per-token alpha tracking reveals local synchronization that decays exponentially with layer distance, and is weaker for reasoning than factual tasks; (6)~Reasoning Step Spectral Punctuation -- phase-transition signatures align with reasoning step boundaries; and (7)~Spectral Correctness Prediction -- spectral α alone achieves AUC = 1.000 (Qwen2.5-7B, late layers) and mean AUC = 0.893 across 6 models in predicting correctness before the final answer is generated. Together, these findings establish a comprehensive spectral theory of reasoning in transformers, revealing that the geometry of thought is universal in direction, architecture-specific in dynamics, and predictive of outcome.