# Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One

arXiv: https://arxiv.org/abs/2606.25449 · HTML: https://mlsanity.com/paper/2606.25449/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 33 references that are in the map.
- First posted on arXiv 24 Jun 2026.

## Builds on (most cited first, 26 of 26)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [C-Pack: Packed Resources For General Chinese Embeddings](https://mlsanity.com/paper/2309.07597/) (2023)
- [MemoryBank: Enhancing Large Language Models with Long-Term Memory](https://mlsanity.com/paper/2305.10250/) (2023)
- [Fast Model Editing at Scale](https://mlsanity.com/paper/2110.11309/) (2021)
- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [Learning to Compress Prompts with Gist Tokens](https://mlsanity.com/paper/2304.08467/) (2023)
- [Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack](https://mlsanity.com/paper/2404.01833/) (2024)
- [Editing Large Language Models: Problems, Methods, and Opportunities](https://mlsanity.com/paper/2305.13172/) (2023)
- [Adapting Language Models to Compress Contexts](https://mlsanity.com/paper/2305.14788/) (2023)
- [AgentTuning: Enabling Generalized Agent Abilities for LLMs](https://mlsanity.com/paper/2310.12823/) (2023)
- [LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models](https://mlsanity.com/paper/2310.05736/) (2023)
- [Do Large Language Models Know What They Don't Know?](https://mlsanity.com/paper/2305.18153/) (2023)
- [When Context Flips, Safety Breaks: Diagnosing Brittle Safety in Aligned Language Models](https://mlsanity.com/paper/2605.27851/) (2026)

## Built on by (most cited first, 7 of 7)

- [Manufactured Confidence: How Memory Consolidation Turns Hearsay into Confident Facts](https://mlsanity.com/paper/2606.29279/) (2026)
- [They Infer What You Meant: Models Represent Communicative Intent More Reliably Than They Act On It](https://mlsanity.com/paper/2607.03598/) (2026)
- [CrystalMem: Elastic Memory for Self-Evolving LLM Agents via Knowledge Crystallization](https://mlsanity.com/paper/2608.00303/) (2026)
- [FACTWASH: Catching AI Rewrites That Wash Hearsay into Fact](https://mlsanity.com/paper/2608.03372/) (2026)
- [What Does Context Compression Cost an Agent? Interaction Costs Unrevealed by Task-Completion Metrics](https://mlsanity.com/paper/2608.16370/) (2026)
- [Re-derivability Decides What a Staged Agent Pipeline Recovers After an Upstream Fault](https://mlsanity.com/paper/2609.32802/) (2026)
- [The Decomposition Tax: LLM Pipelines Lose Up to 40 Accuracy Points at Their Own Interfaces](https://mlsanity.com/paper/2609.32825/) (2026)

## Abstract (arXiv)

A language model's memory can be worse than no memory at all when the model or its interface is disposed to act on it: a memory that keeps a wrong conclusion but drops the work behind it leads a model to re-emit the stale value as a confident answer, where an empty memory leads it to abstain. We call this brittle memory. The information loss is definitional; the finding is behavioral, and it turns on one thing, whether the memory kept a re-derivation basis (the source) rather than the answer. We measure it with reclaim evaluation: induce a known drift, compress at a fixed budget, deliver a correction that names the error, and score exact recovery, judge-free. Holding the budget fixed and varying only what the compression keeps isolates correctability from capability and from size; an 8B model and a frontier one wall in the same place. A one-line source-first policy, keep the recomputable source, drop the re-derivable conclusion, restores correctability at equal budget where the source is compact and identifiable, with a length-matched control ruling out "more text." We map where the fix fails, show the failure compounds through memory loops, and replicate across three deployed memory systems, real dialogue (MultiWOZ), and tau-bench, a deployed-agent benchmark where whether a lossy memory becomes a harmful action is a joint model-and-interface property. We release the harness, the paired memory conditions, and validators built to come out false.