mlsanityData to 14 Oct 2026

MIRAGE: How Conversation State Shapes Historical Evidence Use in Multimodal Personal Agents

arXiv 2609.19059 ↗First posted 25 Aug 2026cs.CL cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

also in 1 more
Abstract (arXiv)

Multimodal large language model (MLLM) agents are increasingly used as personal assistants for long-running tasks. Their utility depends on continuity: agents must retrieve and use earlier evidence across dialogue, files, and workspace state. However, agents can generate plausible answers even when access to that history has degraded, causing outcome-only evaluation to overestimate true evidence use. We present MIRAGE (Multimodal Interaction Retrieval, Attribution, and Grounding Evaluation), a controlled study of historical evidence use under conversation-state variation in multimodal personal agents. MIRAGE holds evidence objects, questions, and scoring fixed while varying only conversation state, and evaluates whether an agent can determine answerability, recover the correct source, and answer from it. Across seven frontier and open-weight multimodal backbones, we find that: 1) pre-compaction depth and post-compaction continuation form distinct, non-monotonic failure regimes rather than a single degradation curve; 2) open-weight models rely heavily on context continuity and are reluctant to spontaneously switch to tool-mediated retrieval when provenance fails; and 3) retrieval pressure improves source attribution in deep pre-compaction states for tool-compliant models, but consistently regresses after compaction, where stored evidence has already degraded. These findings show that historical evidence use should be evaluated under state variation, rather than inferred from outcome-only correctness.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 41 references that are in the map. First posted on arXiv 25 Aug 2026.

Builds on

20 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents7 Jan

Other references · 19

  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  2. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  3. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  4. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  5. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
  6. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
  7. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 367
  8. 2025Context Length Alone Hurts LLM Performance Despite Perfect Retrievalcited by 127
  9. 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memorycited by 97
  10. 2026MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Taskscited by 84
  11. 2025HaluMem: Evaluating Hallucinations in Memory Systems of Agentscited by 54
  12. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 50
7 more
  1. 2025REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understandingcited by 16
  2. 2025Evaluating Long-Context Reasoning in LLM-Based WebAgentscited by 15
  3. 2026SafeHarness: Lifecycle-Integrated Security Architecture for LLM-based Agent Deploymentcited by 15
  4. 2024Revisiting Multimodal Fusion for 3D Anomaly Detection from an Architectural Perspectivecited by 12
  5. 2026Safeguarding LLM Agents against Long-Horizon Threats via Shadow Memorycited by 7
  6. 2026Seeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncationcited by 5
  7. 2026Do LLMs Know Tool Irrelevance? Demystifying Structural Alignment Bias in Tool Invocationscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Propose, Verify, Commit: Evidence-Grounded Memory for Long-Horizon Multi-Actor Conversations
  2. 2026M3Exam: Benchmarking Multimodal Memory for Realistic User-Agent Interactions
  3. 2026TRACE: State-Aware Query Processing over Temporal Evidence Graphs for Conversational Data
  4. 2026LongHarness: Advancing Agentic Reasoning over 100M-Token Corpora via Stateful Evidence Management
  5. 2026MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory
  6. 2026WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction
  7. 2026Ground Truth First: A Longitudinal Evaluation Instrument for Agent Memory, and the Tenure Crossover in Memory-Architecture Rankings
  8. 2025A Simple Yet Strong Baseline for Long-Term Conversational Memory of LLM Agents