# MINTEval: Evaluating Memory under Multi-Target Interference in Long-Horizon Agent Systems arXiv: https://arxiv.org/abs/2605.18565 ยท HTML: https://mlsanity.com/paper/2605.18565/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. - Builds on 25 of its 32 references that are in the map. - First posted on arXiv 18 May 2026. ## Placed in - LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/) ## Builds on (most cited first, 25 of 25) - [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020) - [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024) - [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024) - [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024) - [Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks](https://mlsanity.com/paper/1502.05698/) (2015) - [Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions](https://mlsanity.com/paper/2507.05257/) (2025) - [MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent](https://mlsanity.com/paper/2507.02259/) (2025) - [MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents](https://mlsanity.com/paper/2506.15841/) (2025) - [Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning](https://mlsanity.com/paper/2508.19828/) (2025) - [From RAG to Memory: Non-Parametric Continual Learning for Large Language Models](https://mlsanity.com/paper/2502.14802/) (2025) - [BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack](https://mlsanity.com/paper/2406.10149/) (2024) - [SimpleMem: Efficient Lifelong Memory for LLM Agents](https://mlsanity.com/paper/2601.02553/) (2026) - [HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model](https://mlsanity.com/paper/2408.09559/) (2024) - [Lifelong Learning of Large Language Model based Agents: A Roadmap](https://mlsanity.com/paper/2501.07278/) (2025) - [LifelongAgentBench: Evaluating LLM Agents as Lifelong Learners](https://mlsanity.com/paper/2505.11942/) (2025) - [Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMs](https://mlsanity.com/paper/2510.27246/) (2025) - [Towards Lifelong Dialogue Agents via Timeline-based Memory Management](https://mlsanity.com/paper/2406.10996/) (2024) - [A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents](https://mlsanity.com/paper/2602.06052/) (2026) - [REMem: Reasoning with Episodic Memory in Language Agent](https://mlsanity.com/paper/2602.13530/) (2026) - [StoryBench: A Dynamic Benchmark for Evaluating Long-Term Memory with Multi Turns](https://mlsanity.com/paper/2506.13356/) (2025) - [HorizonBench: Long-Horizon Personalization with Evolving Preferences](https://mlsanity.com/paper/2604.17283/) (2026) - [AtomMem : Learnable Dynamic Agentic Memory with Atomic Memory Operation](https://mlsanity.com/paper/2601.08323/) (2026) - [SYNAPSE: Empowering LLM Agents with Episodic-Semantic Memory via Spreading Activation](https://mlsanity.com/paper/2601.02744/) (2026) - [ES-MemEval: Benchmarking Conversational Agents on Personalized Long-Term Emotional Support](https://mlsanity.com/paper/2602.01885/) (2026) - [MultiSessionCollab: Learning User Preferences with Memory to Improve Long-Term Collaboration](https://mlsanity.com/paper/2601.02702/) (2026) ## Built on by (most cited first, 5 of 5) - [The Horizon Gap: Planning, Memory, Execution, Training, and Evaluation for Long-Horizon LLM Agents](https://mlsanity.com/paper/2608.06663/) (2026) - [Positions Are Not Facts: The Mismatch Between KV Caches and Memory](https://mlsanity.com/paper/2609.33759/) (2026) - [FlowState: Execution State as Memory for Long-Horizon LLM Agents](https://mlsanity.com/paper/2609.34565/) (2026) - [Balancing Memory Pathways: Analyzing and Improving Memory Utilization in Hybrid LMs](https://mlsanity.com/paper/2610.06750/) (2026) - [Beyond Retrieval: Evidence Access and Event Aggregation in Agent Memory](https://mlsanity.com/paper/2610.19474/) (2026) ## Abstract (arXiv) Real-world agents operate over long and evolving horizons, where information is repeatedly updated and may interfere across memories, requiring accurate recall and aggregated reasoning over multiple pieces of information. However, existing benchmarks focus on static, independent recall and fail to capture these dynamic interactions between evolving memories. In this paper, we study how current memory-augmented agents perform in realistic, interference-heavy, long-horizon settings across diverse domains and question types. We introduce MINTEval (Long-Horizon Memory under INTerference Evaluation), a benchmark featuring (1) long, highly interconnected contexts with frequently updated information that induces substantial interference, (2) diverse domains (state tracking, multi-turn dialogue, Wikipedia revisions, and GitHub commits), enabling evaluation of domain generalization, and (3) diverse question types that assess robustness to interference, including (i) single-target recall tasks requiring retrieval of a specific target from long contexts, and (ii) multi-target aggregation tasks requiring reasoning over multiple relevant pieces of information. Overall, MINTEval has 15.6k question-answering pairs over long-horizon contexts averaging 138.8k tokens and extending up to 1.8M tokens per instance. We evaluate 7 representative systems, including vanilla long-context LLMs, RAG, and memory-augmented agent frameworks. Across all systems, we observe consistently low performance (avg. 27.9% accuracy), especially on questions requiring aggregated reasoning over multiple pieces of evidence. Our analysis shows that performance is primarily limited by retrieval and memory construction. Furthermore, current memory systems struggle to recall and reason over earlier facts that are revised or interfered with by subsequent context, with accuracy degrading as the number of intervening updates increases.