The Geometric Price of Discrete Logic: Context-driven Manifold Dynamics of Number Representations
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Large language models (LLMs) generalize smoothly across continuous semantic spaces, yet strict logical reasoning demands the formation of discrete decision boundaries. Prevailing theories relying on linear isometric projections fail to resolve this fundamental tension. In this work, we argue that task context operates as a non-isometric dynamical operator that enforces a necessary "topological distortion." By applying Gram-Schmidt decomposition to residual-stream activations , we reveal a dual-modulation mechanism driving this process: a class-agnostic topological preservation that anchors global structure to prevent semantic collapse, and a specific algebraic divergence that directionally tears apart cross-class concepts to forge logical boundaries. We validate this geometric evolution across a gradient of tasks, from simple mapping to complex primality testing. Crucially, targeted specific vector ablation establishes a strict causal binding between this topology and model function: algebraically erasing the divergence component collapses parity classification accuracy from 100% to chance levels (38.57%). Furthermore, we uncover a three-phase layer-wise geometric dynamic and demonstrate that under social pressure prompts, models fail to generate sufficient divergence. This results in a "manifold entanglement" that geometrically explains sycophancy and hallucination. Ultimately, our findings revise the linear-isometric presumption, demonstrating that the emergence of discrete logic in LLMs is purchased at an irreducible cost of topological deformation.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 11 of its 13 references that are in the map. First posted on arXiv 24 Mar 2026.
Builds on
11 of 13 references.
Closest ancestors · 1
Same direction, newest first
Other references · 10
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
- 2019Root Mean Square Layer Normalizationcited by 1,425
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
- 2025Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometrycited by 58
- 2025Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learningcited by 10
- 2025Geometry of Decision Making in Language Modelscited by 8
- 2026LSSF: Safety Alignment for Large Language Models through Low-Rank Safety Subspace Fusioncited by 8
- 2026Low-Dimensional Execution Manifolds in Transformer Learning Dynamics: Evidence from Modular Arithmetic Taskscited by 7
- 2025Human-like Social Compliance in Large Language Models: Unifying Sycophancy and Conformity through Signal Competition Dynamicscited by 4
- 2026The Representational Geometry of Numbercited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Emergent Manifold Separability during Reasoning in Large Language Models
- 2026Reasoning emerges from constrained inference manifolds in large language models
- 2026Unveiling the Reasoning Process of Large Language Models
- 2026The Geometry of Thought: How Scale Restructures Reasoning In Large Language Models
- 2026Do Large Language Models Play Six Degrees of Separation? Measuring Topological Compression in Long-Context Manifolds
- 2026Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs
- 2026Context Is King: How In-Context Specification Shapes the Geometry of Concepts
- 2026Encoded but Not Actionable: Auditing the Decode-Generate-Steer Gap in Frozen LLMs for Geometric Constraints