mlsanityData to 14 Oct 2026

Compact-Memory LLM Agents via Online Max-Member Clustering and Atom-Aware Packing

arXiv 2609.04915 ↗First posted 4 Sep 2026cs.AIcited by 0

On the map: LLM agents and tool use → Agent memory systems

Abstract (arXiv)

Many long-horizon LLM deployments face tight prompt budgets: latency, cost, and context limits make full-context prompting impractical as interaction length grows. The key question is then not raw recall alone, but which memory design gives the best quality--token trade-off in the compact-memory regime. We present RSM-full, an online clustered-memory pipeline designed for a strong quality--token Pareto point. RSM-full combines two design choices: a cosine-gated max-member merge write rule and an atom-aware grouped context packer. On AMA-Bench, our primary compact-memory benchmark, it reaches 83% of Full-Context quality at 32% of the token cost at a 4k budget; under four-seed averaging it beats the closest streaming-clustered baseline (Online K-Means) by +3.5--6.0,pp (p<.001) across the whole ~2.6k--~5k regime. Three-seed ablations show most of this gain comes from the merge rule (+5.7,pp over Online K-Means and matched-τ DP-means) and the grouped packer (+5.0,pp over flat concatenation). The pattern reproduces on RealMem, an independent long-horizon persona-memory benchmark: RSM-full improves on Budget-RAG (+0.69,pp, p=.006), is on par with BM25-RAG (paired Delta=+0.27,pp, p=.47; we do not claim BM25 equivalence in the equivalence-test sense), and significantly outperforms Streaming-Proto (+2.97,pp) and the closest reproduced 2025 agentic-memory baseline A-MEM (+1.65,pp, p<.001). Across benchmarks the message is consistent: under tight budgets, compact-memory performance is driven mainly by how streaming memories are merged and how retrieved content is assembled. Overall, RSM-full is most useful when answeroughly 2k--5k prompt tokens, where itdefines a strong compact-memory Pareto point; higher-token baselines remain stronger outside this regime.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 17 of its 22 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

17 of 22 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026Memori: A Persistent Memory Layer for Efficient, Context-Aware LLM Agents20 Mar
  2. 2026Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers8 Mar
  3. 2026AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications26 Feb
  4. 2026Active Context Compression: Autonomous Memory Management in LLM Agents12 Jan
  5. 2025ACON: Optimizing Context Compression for Long-horizon LLM Agents1 Oct
  6. 2025A-MEM: Agentic Memory for LLM Agents17 Feb
  7. 2024HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model18 Aug
  8. 2024Evaluating Very Long-Term Conversational Memory of LLM Agents27 Feb

Other references · 9

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  5. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  6. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  7. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 556
  8. 2025From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMscited by 124
  9. 2026Latent Context Compilation: Distilling Long Context into Compact Portable Memorycited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026TokenPilot: Cache-Efficient Context Management for LLM Agents
  2. 2026Akashic: A Low-Overhead LLM Inference Service with MemAttention
  3. 2026Shared Selective Persistent Memory for Agentic LLM Systems
  4. 2026Evaluating Memory Structure in LLM Agents
  5. 2026Pancake: Hierarchical Memory System for Multi-Agent LLM Serving
  6. 2026Memory as Infrastructure: Reliability Engineering for Persistent Agent Memory in Months-Long LLM-Assisted Development
  7. 2026MemCalib: Benchmarking and Optimizing Memory Use in LLM Agents
  8. 2026SimpleMem: Efficient Lifelong Memory for LLM Agents