mlsanityData to 7 Oct 2026

CoreMem: Riemannian Retrieval and Fisher-Guided Distillation for Long-Term Memory in Dialogue Agents

arXiv 2606.18406 ↗First posted 16 Jun 2026cs.CLcited by 1

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Personalized dialogue agents require continuous long-term memory to maintain coherent interactions across multiple sessions. However, deploying these capabilities on consumer-grade hardware (e.g., 8 GB VRAM edge devices) introduces severe memory and compute bottlenecks. Existing systems typically rely on isotropic cosine similarity for retrieval and heuristic rules for context compression. These approaches lack a unified theoretical foundation, frequently suffering from the hubness problem in high-dimensional retrieval and syntactic fragmentation during compression. To overcome these limitations, we propose CoreMem, a resource-efficient edge-cloud memory architecture fundamentally unified by information geometry. First, Riemannian retrieval replaces cosine matching with a locally adaptive Fisher-Rao metric, effectively penalizing hub memories via Mahalanobis distance with O(Ndr) Woodbury acceleration for real-time search. Second, Fisher-guided discrete token distillation (FDTD) introduces a hierarchical sentence-to-token compression mechanism. It derives sensitivity scores from Fisher information traces, providing a principled compression-KL tradeoff augmented with explicit structural syntax protection. Evaluated on the LOCOMO and LongMemEval-S benchmarks, CoreMem achieves strong accuracy improvements, yielding substantial gains in Open-domain (+4.51 pp) and Temporal (+4.17 pp) reasoning. Extensive profiling confirms that CoreMem operates seamlessly within a strict 8 GB VRAM budget, successfully bridging the gap between resource-constrained edge devices and the demand for theoretically grounded, lifelong memory agents.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 20 of its 37 references that are in the map. First posted on arXiv 16 Jun 2026.

Builds on

20 of 37 references.

References · 20

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  4. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  5. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  6. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
  7. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
  8. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  9. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  10. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  11. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
  12. 2023Learning to Compress Prompts with Gist Tokenscited by 379
8 more
  1. 2023Adapting Language Models to Compress Contextscited by 350
  2. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 343
  3. 2019Limitations of the Empirical Fisher Approximation for Natural Gradient Descentcited by 213
  4. 2024Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilizationcited by 97
  5. 2024Prompt Compression for Large Language Models: A Surveycited by 65
  6. 2025Context Cascade Compression: Exploring the Upper Limits of Text Compressioncited by 17
  7. 2025CarMem: Enhancing Long-Term Memory in LLM Voice Assistants through Category-Boundingcited by 8
  8. 2025LexSemBridge: Fine-Grained Dense Representation Enhancement through Token-Aware Embedding Augmentationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Hiding in Plain Sight: A Diffusion-based Mitigation of Geolocation Privacy Leakage in Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MemMachine: A Ground-Truth-Preserving Memory System for Personalized AI Agents
  2. 2026Beyond Memory Construction: Rethinking Memory Access for LLM-based Conversational Agents
  3. 2026MemRefine: LLM-Guided Compression for Long-Term Agent Memory
  4. 2026SuperLocalMemory V3: Information-Geometric Foundations for Zero-LLM Enterprise Agent Memory
  5. 2026PRISM: Pareto-Efficient Retrieval over Intent-Aware Structured Memory for Long-Horizon Agents
  6. 2026AgentMemBench: A Systematic Benchmark for Evaluating Long-Term Memory Management Strategies in Conversational AI Agents
  7. 2026G-Long: Graph-Enhanced Memory Management for Efficient Long-Term Dialogue Agents
  8. 2026SimpleMem: Efficient Lifelong Memory for LLM Agents