mlsanityData to 14 Oct 2026

Do Large Language Models Play Six Degrees of Separation? Measuring Topological Compression in Long-Context Manifolds

arXiv 2608.17950 ↗First posted 18 Aug 2026cs.CLcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large Language Models (LLMs) demonstrate remarkable multi-hop reasoning capabilities over long contexts, yet the internal mechanisms enabling these distant cognitive leaps remain poorly understood. Traditional attention-based interpretability often fails to capture true semantic proximity due to routing artifacts like attention sinks. In this paper, we bypass attention weights to directly analyze the dynamic geometry of the hidden state manifold, proving that deep LLM latent spaces natively organize into Small-World networks. By sparsifying the continuous similarity matrices of long-context representations into unweighted graphs, we trace the connectivity between highly disjoint semantic anchors across two distinct architectures. Our findings reveal a sharp topological phase transition: while early syntactic layers remain entirely fractured, deep reasoning layers abruptly compress massive conceptual distances into highly navigable pathways strictly bounded by the "Six Degrees of Separation" limit (=< 6 semantic hops). Furthermore, we demonstrate the practical efficacy of this framework by applying it to zero-shot hallucination detection within Retrieval-Augmented Generation (RAG) using the RAGognize dataset. We show that factually grounded generations maintain structural integrity with their source context (approximately 3 hops), whereas hallucinations induce severe topological collapse. Ultimately, this work mathematically formalizes how transformers execute abstract reasoning and provides a novel, strictly geometric signature for evaluating factual reliability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 10 of its 19 references that are in the map. First posted on arXiv 18 Aug 2026.

Builds on

10 of 19 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025The Shape of Reasoning: Topological Analysis of Reasoning Traces in Large Language Models23 Oct

Other references · 9

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Pointer Sentinel Mixture Modelscited by 3,096
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  4. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  5. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
  6. 2020Quantifying Attention Flow in Transformerscited by 775
  7. 2019Revealing the Dark Secrets of BERTcited by 301
  8. 2026The Topological Trouble With Transformerscited by 3
  9. 2026RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integrationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations
  2. 2025Large Language Models Encode Semantics and Alignment in Linearly Separable Representations
  3. 2026The Geometric Price of Discrete Logic: Context-driven Manifold Dynamics of Number Representations
  4. 2026Emergent Manifold Separability during Reasoning in Large Language Models
  5. 2025Demystifying the Roles of LLM Layers in Retrieval, Knowledge, and Reasoning
  6. 2025REMA: A Unified Reasoning Manifold Framework for Interpreting Large Language Model
  7. 2025Hallucination Detection in LLMs with Topological Divergence on Attention Graphs
  8. 2026Beyond Linear Concepts: Discovering and Aligning Non-Linear Concept Manifolds in Large Language Models