# Manufactured Confidence: How Memory Consolidation Turns Hearsay into Confident Facts

arXiv: https://arxiv.org/abs/2606.29279 · HTML: https://mlsanity.com/paper/2606.29279/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 9 of its 16 references that are in the map.
- First posted on arXiv 28 Jun 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 9 of 9)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Ignore Previous Prompt: Attack Techniques For Language Models](https://mlsanity.com/paper/2211.09527/) (2022)
- [Zep: A Temporal Knowledge Graph Architecture for Agent Memory](https://mlsanity.com/paper/2501.13956/) (2025)
- [Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts](https://mlsanity.com/paper/2305.13300/) (2023)
- [Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One](https://mlsanity.com/paper/2606.25449/) (2026)
- [When Context Flips, Safety Breaks: Diagnosing Brittle Safety in Aligned Language Models](https://mlsanity.com/paper/2605.27851/) (2026)

## Built on by (most cited first, 3 of 3)

- [FACTWASH: Catching AI Rewrites That Wash Hearsay into Fact](https://mlsanity.com/paper/2608.03372/) (2026)
- [Memory Consolidation Flattens the Temporal Shape of User Facts](https://mlsanity.com/paper/2609.36457/) (2026)
- [When to Remember, When to Abstain: Category-Conditioned Retention for Reliable Agent Memory](https://mlsanity.com/paper/2610.07100/) (2026)

## Abstract (arXiv)

LLM agents carry conclusions across steps and sessions in compressed memory, and memory products (e.g., mem0, LangMem) rewrite conversation into stored "facts" that later steps trust. We show this rewriting manufactures confidence: across our constructed agent settings, a casual, hedged remark becomes a confident, dated assertion the agent then obeys like a verified fact, granting every above-clearance request it faces. No attacker is needed: a role that was true once and never corrected is stored as a flat fact and acted on like a deliberate injection. We then isolate what the agent responds to. It is not the source: attributed, unattributed, and even forged "system of record" claims all grant alike. It is the confidence of the phrasing. A hedge is discounted, a flat assertion is obeyed, and this holds with no special keyword. Not all hedges are equal, though: the evidential register is the least-discounted, with "reportedly" obeyed like a flat assertion on most models. The obvious fixes fail. A passive "unverified" tag is ignored, and an active "do not trust this" instruction escalates even correct memory, so it is safe only by refusing to decide. The real fix lives in the store: keep the tentative phrasing rather than upgrade it. But that is hygiene, not a defense against an attacker who can simply write a confident lie. The deployable lesson is narrower and constructive: a single load-bearing memory is the hazard, and one redundant source restores correct decisions. We release the harness and demonstrations.