ATANT v1.1: Positioning Continuity Evaluation Against Memory, Long-Context, and Agentic-Memory Benchmarks
On the map: LLM agents and tool use → Agent memory systems
Abstract (arXiv)
ATANT v1.0 (arXiv:2604.06710) defined continuity as a system property with 7 required properties and introduced a 10-checkpoint, LLM-free evaluation methodology validated on a 250-story corpus. Since publication, a recurring reviewer and practitioner question has concerned not the framework itself but its relationship to a wider set of memory evaluations: LOCOMO, LongMemEval, BEAM, MemoryBench, Zep's evaluation suite, Letta/MemGPT's evaluations, and RULER. This companion paper, v1.1, does not modify the v1.0 standard. It closes a related-work gap that v1.0 left brief under page limits. We show by structural analysis that none of these benchmarks measures continuity as defined in v1.0: of the 7 required properties, the median existing eval covers 1 property, the mean covers 0.43 when partial credit is scored at 0.5, and no eval covers more than 2. We provide a cell-by-cell property-coverage matrix, identify methodological defects specific to each benchmark (including an empty-gold scoring bug in the LOCOMO reference implementation that renders 23% of its corpus unscorable by construction), and publish our reference implementation's LOCOMO score (8.8%) alongside the structural reason that number is uninformative about continuity. We publish our 8.8% LOCOMO score alongside our 96% ATANT cumulative-scale score as a calibration pair: the 87-point divergence is evidence that the two benchmarks measure different properties, not that one system is an order of magnitude better than another. The position v1.1 takes is not adversarial: each benchmark measures a real capability. The claim is that none of them can adjudicate continuity, and conflating them with continuity evaluation has led the field to under-invest in the properties v1.0 names.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 9 of its 10 references that are in the map. First posted on arXiv 13 Apr 2026.
Builds on
9 of 10 references.
Closest ancestors · 3
Same direction, newest first
- 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory28 Apr
- 2025Zep: A Temporal Knowledge Graph Architecture for Agent Memory20 Jan
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agents27 Feb
Other references · 6
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
- 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
- 2025MemoryBench: A Benchmark for Memory and Continual Learning in LLM Systemscited by 61
- 2025Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMscited by 60
- 2026ATANT: An Evaluation Framework for AI Continuitycited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026VitaBench: Memory Lifecycles Beyond Retrieval
- 2026When Retrieval Fails Before It Begins: Structurally Indirect Prerequisite Eviction as a Retention Failure in Agentic Memory
- 2026Memory Canonicalization: A Framework and Benchmark for Cross-Model Drift in Persistent LLM Memory
- 2026Is One Score Enough? Rethinking the Evaluation of Sequentially Evolving LLM Memory
- 2026Continuity Laws for Sequential Models
- 2026MemoryLake on MemoryArena: A Matched Study of Agent Memory Backends
- 2026MemGuard: Persisting Verifier Signals for LLM-Agent Memory Governance
- 2026MemDelta: Controlled Baselines and Hidden Confounds in Agent Memory Evaluation