TokenMizer: Graph-Structured Session Memory for Long-Horizon LLM Context Management
Not on the map. Why?
Abstract (arXiv)
Long-horizon LLM sessions outlive their context windows, and the standard mitigations - truncation, summarization, retrieval - share a structural flaw: they treat history as flat text, discarding precisely the content that makes a session resumable: decisions and their rationales, task status, and file modification history. We present TokenMizer, an open-source transparent proxy that maintains session history as a typed knowledge graph and, at context boundaries, replaces the raw transcript with a token-budgeted serialization of session state. The schema comprises 14 node types and 7 edge types under an 8-state lifecycle in which decisions can be superseded or explicitly invalidated; bitemporal validity intervals support time-travel queries; and first-class decision-transition records preserve why each decision replaced its predecessor (trigger, reason, evidence). Version 0.3.1 embeds this memory core in a production-shaped serving layer - SSE streaming, security middleware, nine provider adapters, a monitoring dashboard, graph exports (D3 JSON, self-contained interactive HTML, Obsidian Canvas) - and exposes checkpoint/resume to agents as Model Context Protocol tools. The evaluation is deliberately minimal and fully provenanced: three synthetic sessions, heuristic-only extraction, one plain-summary baseline, every value traceable to a single versioned results file. Graph extraction ties the baseline on task recall (75.6%) and exceeds it on decision recall (85.0% vs. 70.0%) and file recall (100% vs. 91.7%), with 201-302-token resume blocks extracted in 8.1-529.9 ms per session. At n=3 these results are directional; ceiling effects and baseline weaknesses are analyzed explicitly. Code, benchmark runner, and the exact results file are released under the MIT licence.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 31 references that are in the map. First posted on arXiv 4 Jun 2026.
Builds on
12 of 31 references.
References · 12
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
- 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 453
- 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 399
- 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 287
- 2025Context Is What You Need: The Maximum Effective Context Window for Real World Limits of LLMscited by 12
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Lost in Context Compaction: When LLM Systems Drop Side Constraints
- 2026Beyond Compaction: Structured Context Eviction for Long-Horizon Agents
- 2026Contextual Memory Virtualisation: DAG-Based State Management and Structurally Lossless Trimming for LLM Agents
- 2026Persistent Context Graphs for Efficient Memory Compaction in LLM Agents
- 2026Shared Selective Persistent Memory for Agentic LLM Systems
- 2026Less Context, More Accuracy: A Bi-Temporal Memory Engine for LLM Agents Where a Lean Retrieved Context Beats the Full History
- 2026LLM Agents Are Latent Context Managers: Eliciting Self-Managed Context via State Proprioception
- 2026REAL: A Reasoning-Enhanced Graph Framework for Long-Term Memory Management of LLMs