Context Cartography: Toward Structured Governance of Contextual Space in Large Language Model Systems
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
The prevailing approach to improving large language model (LLM) reasoning has centered on expanding context windows, implicitly assuming that more tokens yield better performance. However, empirical evidence - including the "lost in the middle" effect and long-distance relational degradation - demonstrates that contextual space exhibits structural gradients, salience asymmetries, and entropy accumulation under transformer architectures. We introduce Context Cartography, a formal framework for the deliberate governance of contextual space. We define a tripartite zonal model partitioning the informational universe into black fog (unobserved), gray fog (stored memory), and the visible field (active reasoning surface), and formalize seven cartographic operators - reconnaissance, selection, simplification, aggregation, projection, displacement, and layering - as transformations governing information transitions between and within zones. The operators are derived from a systematic coverage analysis of all non-trivial zone transformations and are organized by transformation type (what the operator does) and zone scope (where it applies). We ground the framework in the salience geometry of transformer attention, characterizing cartographic operators as necessary compensations for linear prefix memory, append-only state, and entropy accumulation under expanding context. An analysis of four contemporary systems (Claude Code, Letta, MemOS, and OpenViking) provides interpretive evidence that these operators are converging independently across the industry. We derive testable predictions from the framework - including operator-specific ablation hypotheses - and propose a diagnostic benchmark for empirical validation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 68 of its 93 references that are in the map. First posted on arXiv 21 Mar 2026.
Builds on
68 of 93 references.
References · 68
- 2017Attention Is All You Needcited by 60,361
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2020Longformer: The Long-Document Transformercited by 3,413
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
38 more
- 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
- 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
- 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
- 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
- 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
- 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 431
- 2023Cognitive Architectures for Language Agentscited by 429
- 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 401
- 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 398
- 2024Titans: Learning to Memorize at Test Timecited by 328
- 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
- 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 313
- 2025Memory in the Age of AI Agentscited by 303
- 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 281
- 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 265
- 2025RWKV-7 "Goose" with Expressive Dynamic State Evolutioncited by 141
- 2025MemOS: A Memory OS for AI Systemcited by 138
- 2025ACON: Optimizing Context Compression for Long-horizon LLM Agentscited by 136
- 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!cited by 132
- 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 131
- 2025A Survey of Context Engineering for Large Language Modelscited by 126
- 2025Recursive Language Modelscited by 104
- 2024Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilizationcited by 101
- 2025How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behaviorcited by 101
- 2025Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Modelscited by 100
- 2025Self-Adapting Language Modelscited by 84
- 2025MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Modelscited by 73
- 2024Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encodingcited by 72
- 2023Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Modelscited by 62
- 2025On the Emergence of Position Bias in Transformerscited by 59
- 2025The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Managementcited by 48
- 2024When Large Language Models Meet Vector Databases: A Surveycited by 39
- 2025AgentOrchestra: Orchestrating Multi-Agent Intelligence with the Tool-Environment-Agent(TEA) Protocolcited by 39
50 most cited of 68 All 68 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Seeing, Saying, but Not Using: From Reportable Spatial Facts to Usable States in Multimodal Large Language Models
- 2026PEEK: Context Map as an Orientation Cache for Long-Context LLM Agents
- 2026State Design Matters: How Representations Shape Dynamic Reasoning in Large Language Models
- 2026Unveiling the Reasoning Process of Large Language Models
- 2024Insights into LLM Long-Context Failures: When Transformers Know but Don't Tell
- 2025Stuck in the Matrix: Probing Spatial Reasoning in Large Language Models
- 2025Linear Spatial World Models Emerge in Large Language Models
- 2026From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs