mlsanityData to 14 Oct 2026

Hashrope: A Persistent Data Structure for Mechanisms of LLM Context Management

arXiv 2610.16144 ↗First posted 12 Oct 2026cs.DS cs.CLcited by 0

On the map: Efficient LLM inference and serving → Efficient reasoning and agent inference

Abstract (arXiv)

Serving stacks for large language models manage context through four recurring mechanisms: incremental edits to a growing context, branching and snapshotting for tree search, identification of reusable prefixes for KV-cache reuse, and encoding of repeated segments. Each mechanism is served today by a different specialist structure. We show that all four reduce to a single persistent weight-balanced rope whose every node carries a polynomial hash of its subtree over a Mersenne prime, so that the structure supports logarithmic edits and logarithmic verified content-identity queries at the same time. We validate each mechanism with pre-registered, replicated experiments, then compare against a deployed specialist per mechanism: the Ropey editing library, a faithful PagedAttention copy-on-write block table, and the vendored SGLang radix matcher. On workloads that interleave edits with identity queries, Hashrope is net-superior to Ropey by a factor that grows without bound (369.7x at one million characters). Against the radix matcher on a live vLLM stack serving Qwen2.5-7B-Instruct-1M on four A100 GPUs, Hashrope identifies a one-million-token shared prefix exactly and 1.20x faster, at a context length the dense engine cannot even prefill.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Efficient reasoning and agent inference, one of 12 directions in Efficient LLM inference and serving. Builds on 6 of its 15 references that are in the map. First posted on arXiv 12 Oct 2026.

Builds on

6 of 15 references.

References · 6

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  3. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  4. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 461
  5. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
  6. 2025Qwen2.5-1M Technical Reportcited by 164

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Mergeable Model-Side Aggregation States for Long-Context Language Models
  2. 2026ContextPipe: Database-Inspired Context Assembly for Long-Horizon Agents
  3. 2026Codebase-Memory: Tree-Sitter-Based Knowledge Graphs for LLM Code Exploration via MCP
  4. 2026Asynchronous Verified Semantic Caching for Tiered LLM Architectures
  5. 2026Contextual Memory Virtualisation: DAG-Based State Management and Structurally Lossless Trimming for LLM Agents
  6. 2026PEEK: Context Map as an Orientation Cache for Long-Context LLM Agents
  7. 2026TokenPilot: Cache-Efficient Context Management for LLM Agents
  8. 2026The Labyrinth and the Thread: Rethinking Regularizations in Sequential Knowledge Editing for Large Language Models