Pull: Lazy Materialization of Working Memory for Stateful LLM Conversations
Not on the map. Why?
Abstract (arXiv)
As LLM conversations grow to hundreds of turns, full-context injection incurs O(N2) cumulative token costs, while lossy summarization or hard truncation irreversibly discards historical state. We propose Pull, a session router that maintains an addressable metadata directory via a local, deterministic Purifier (zero LLM calls, millisecond-level latency). At query time, the LLM lazily materializes only the turns it needs; unmaterialized turns remain accessible but collapsed. Unlike irreversible compression, Pull's materialization is reversible; subsequent queries can expand any collapsed turn. On LoCoEval (128 conversations, 12,780 turns), Pull reduces per-query context tokens (Phase 2) by 75.1 percent on single-hop tasks with equivalent quality (Delta = -0.002, n.s.) and by 72.0 percent on multi-hop tasks with no quality loss (Delta = +0.017). A controlled routing benchmark (7,831 queries x 10 methods) shows that entity lifecycle tracking is empirically a prerequisite for distance-independent routing. On BEAM 1M (14 conversations, 263 questions), Pull improves F1 by +55.2 percent over a truncation baseline.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 25 references that are in the map. First posted on arXiv 13 Sep 2026.
Builds on
14 of 25 references.
References · 14
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
- 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
- 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
- 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 439
- 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
- 2025Zep: A Temporal Knowledge Graph Architecture for Agent Memorycited by 400
- 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 348
- 2025Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMscited by 60
Built on by
Not cited yet.