mlsanityData to 14 Oct 2026

Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One

arXiv 2606.25449 ↗First posted 24 Jun 2026cs.CL cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

A language model's memory can be worse than no memory at all when the model or its interface is disposed to act on it: a memory that keeps a wrong conclusion but drops the work behind it leads a model to re-emit the stale value as a confident answer, where an empty memory leads it to abstain. We call this brittle memory. The information loss is definitional; the finding is behavioral, and it turns on one thing, whether the memory kept a re-derivation basis (the source) rather than the answer. We measure it with reclaim evaluation: induce a known drift, compress at a fixed budget, deliver a correction that names the error, and score exact recovery, judge-free. Holding the budget fixed and varying only what the compression keeps isolates correctability from capability and from size; an 8B model and a frontier one wall in the same place. A one-line source-first policy, keep the recomputable source, drop the re-derivable conclusion, restores correctability at equal budget where the source is compact and identifiable, with a length-matched control ruling out "more text." We map where the fix fails, show the failure compounds through memory loops, and replicate across three deployed memory systems, real dialogue (MultiWOZ), and tau-bench, a deployed-agent benchmark where whether a lossy memory becomes a harmful action is a joint model-and-interface property. We release the harness, the paired memory conditions, and validators built to come out false.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 26 of its 33 references that are in the map. First posted on arXiv 24 Jun 2026.

Builds on

26 of 33 references.

References · 26

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  6. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  7. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  8. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  9. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  10. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  11. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  12. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
14 more
  1. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  2. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
  3. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 578
  4. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 556
  5. 2021Fast Model Editing at Scalecited by 498
  6. 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
  7. 2023Learning to Compress Prompts with Gist Tokenscited by 390
  8. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
  9. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  10. 2023Adapting Language Models to Compress Contextscited by 356
  11. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 338
  12. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 287
  13. 2023Do Large Language Models Know What They Don't Know?cited by 243
  14. 2026When Context Flips, Safety Breaks: Diagnosing Brittle Safety in Aligned Language Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.