# Ground Truth First: A Longitudinal Evaluation Instrument for Agent Memory, and the Tenure Crossover in Memory-Architecture Rankings arXiv: https://arxiv.org/abs/2607.21962 ยท HTML: https://mlsanity.com/paper/2607.21962/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. - Builds on 50 of its 59 references that are in the map. - First posted on arXiv 24 Jul 2026. ## Placed in - LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/) ## Builds on (most cited first, 30 of 50) - [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023) - [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023) - [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022) - [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022) - [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023) - [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023) - [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023) - [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025) - [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022) - [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024) - [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025) - [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024) - [MemoryBank: Enhancing Large Language Models with Long-Term Memory](https://mlsanity.com/paper/2305.10250/) (2023) - [AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases](https://mlsanity.com/paper/2407.12784/) (2024) - [Zep: A Temporal Knowledge Graph Architecture for Agent Memory](https://mlsanity.com/paper/2501.13956/) (2025) - [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021) - [MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions](https://mlsanity.com/paper/2305.14795/) (2023) - [HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models](https://mlsanity.com/paper/2405.14831/) (2024) - [Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions](https://mlsanity.com/paper/2507.05257/) (2025) - [Benchmark Data Contamination of Large Language Models: A Survey](https://mlsanity.com/paper/2406.04244/) (2024) - [Know Your Limits: A Survey of Abstention in Large Language Models](https://mlsanity.com/paper/2407.18418/) (2024) - [MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks](https://mlsanity.com/paper/2602.16313/) (2026) - [Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMs](https://mlsanity.com/paper/2510.27246/) (2025) - [From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents](https://mlsanity.com/paper/2606.04329/) (2026) - [Mem2ActBench: A Benchmark for Evaluating Long-Term Memory Utilization in Task-Oriented Autonomous Agents](https://mlsanity.com/paper/2601.19935/) (2026) - [A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecycle](https://mlsanity.com/paper/2604.16548/) (2026) - [Hidden in Memory: Sleeper Memory Poisoning in LLM Agents](https://mlsanity.com/paper/2605.15338/) (2026) - [Anatomy of Agentic Memory: Taxonomy and Empirical Analysis of Evaluation and System Limitations](https://mlsanity.com/paper/2602.19320/) (2026) - [Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents](https://mlsanity.com/paper/2606.27472/) (2026) - [Governing Evolving Memory in LLM Agents: Risks, Mechanisms, and the Stability and Safety Governed Memory (SSGM) Framework](https://mlsanity.com/paper/2603.11768/) (2026) ## Built on by (most cited first, 3 of 3) - [RoutePrism: Tracing Construction Order Effects in Agent Memory](https://mlsanity.com/paper/2609.34160/) (2026) - [When Evidence Changes: Evaluating Memory Repair and Re-reading in Language-Model Agents](https://mlsanity.com/paper/2610.03902/) (2026) - [When Should Enterprise Policy Decisions Be Learned, and When Should They Be Reasoned?](https://mlsanity.com/paper/2610.17643/) (2026) ## Abstract (arXiv) Benchmarks for LLM-agent memory typically generate conversations first and extract answer keys afterwards -- with documented label-error and contamination problems -- and they overwhelmingly measure short interaction histories. We invert the pipeline: a seeded life-script sampler emits facts with validity intervals, volatility classes, and source channels before any text exists; an LLM renderer writes chat and email from per-event fact manifests; a fidelity verifier confirms every planted fact; and questions are instantiated mechanically from the script, so gold answers are script-valid by construction and separately validated for answerability. The synthetic, fictionalized corpus (~380 questions, 15 types) embeds features absent from the benchmarks we survey: per-fact validity intervals, sent/received trust distinctions, injection probes in a benign harness, and as-of-date question sets. Benchmarking five memory architectures against a no-memory control (fixed answerer, versioned LLM judge, three replicates, two horizons), we find backend rankings invert with history length: the budgeted curated-map memory that leads at three weeks loses recall of evicted content by nine weeks (96% to 72%) while a provenance-typed graph rises to 90%; the inversion is positive for all six users under complete cross-family re-judging (exact p=0.031). A full-rendered-history baseline ties or exceeds the best memory system at the short horizon but shows no judge-independent advantage at nine weeks, at about twice the read cost. Write-stage quality strongly correlates with downstream quality (weakly-written facts fail 24% vs 2%), and injection resistance tracked whether provenance boundaries survive representation. A layered architecture performs best among the memory systems in both regimes (96.8% short-horizon) and is released as Veracium, an open-source library, with the corpus generator and harness.