mlsanityData to 7 Oct 2026

Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell

arXiv 2610.07782 ↗First posted 6 Oct 2026cs.AI cs.CL cs.DCcited by 0

On the map: LLM agents and tool use → Agent memory systems

also in 1 more
Abstract (arXiv)

Decomposing long-context inference across cooperating agents bounds the active KV cache per call rather than total evidence, which matters when KV-cache memory binds. Many such systems add a persistent tier storing and recalling reasoning traces, usually validated by an ablation reporting an accuracy gain. We measure both on one three-tier agent architecture. Decomposition delivers: peak KV working set of 14.3 MiB per query against 35.5 and 35.3 MiB for single-pass and retrieval-augmented baselines. The persistent tier does not: across eight controlled dataset pairs at n=100 per arm it costs +0.368 MiB [+0.167, +0.590] of peak cache and produces no detectable accuracy change (+0.015, 95% CI [-0.011, +0.046]). We argue the null is structural: single-question benchmarks supply each item with its own evidence and score it independently, and correctness requires resetting stored traces between conditions, so recall has nothing informative to retrieve. Reaching it took four measurement corrections -- three inflating the apparent benefit, the fourth making an effect that size look resolvable -- none visible in the results table. We give the conditions an agent-memory ablation must satisfy and detection procedures that need no knowledge of the specific defect.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 12 of its 12 references that are in the map. First posted on arXiv 6 Oct 2026.

Builds on

12 of 12 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Evaluating Very Long-Term Conversational Memory of LLM Agents27 Feb

Other references · 11

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  7. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  8. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  9. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  10. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201
  11. 2021Accounting for Variance in Machine Learning Benchmarkscited by 195

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents
  2. 2026Stop When Memory Suffices: Evidence-Conditioned Progressive Execution for LLM Agents
  3. 2026Agent Memory Below the Prompt: Persistent Q4 KV Cache for Multi-Agent LLM Inference on Edge Devices
  4. 2026Evaluating Memory Structure in LLM Agents
  5. 2026The Epistemics of Agent Memory: Measuring, and Governing, the Consolidation Decision in Long-Horizon LLM Agents
  6. 2026STALE: Can LLM Agents Know When Their Memories Are No Longer Valid?
  7. 2026Akashic: A Low-Overhead LLM Inference Service with MemAttention
  8. 2026MemGym: a Long-Horizon Memory Environment for LLM Agents