ICM-Bench: Person-Level Identity Reasoning in Multimodal Agents with Long-Term Memory
Not on the map. Why?
Abstract (arXiv)
Long-horizon multimodal agents should remember not only what happened but also who participated. This capability depends on linking recurring faces, voices, names, person-associated objects, events, and social relations to consistent identities over time. Existing long-video and multimodal-agent benchmarks measure broad memory question answering, but they do not isolate the ability to maintain recurring person identities and reason over their cross-time relations. We introduce ICM-Bench (Identity-Centric Memory Benchmark), which, to the best of our knowledge, is the first benchmark specifically designed to evaluate identity-centric reasoning over long video memories in multimodal agents. The benchmark contains 839 synthetic clips spanning 141 minutes and 1,217 open-ended questions about six recurring adults in a one-year life album. A theme-configurable pipeline generates the video collection and associates each question with its target identities and traceable supporting evidence. We compare direct caption-memory baselines, memory-augmented agents, and graph-retrieval systems. Gemini 3.1 Pro achieves the highest overall accuracy of 74.0%, yet its score falls to 60.3% on questions that require long-term identity profiles. The results show that current systems recover many event-level memories but remain less reliable when evidence must be accumulated around a stable person.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 39 references that are in the map. First posted on arXiv 3 Sep 2026.
Builds on
31 of 39 references.
References · 31
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
- 2025Qwen2.5-Omni Technical Reportcited by 736
- 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
- 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
19 more
- 2019Bag of Tricks and A Strong Baseline for Deep Person Re-identificationcited by 430
- 2025From RAG to Memory: Non-Parametric Continual Learning for Large Language Modelscited by 231
- 2019pyannote.audio: neural building blocks for speaker diarizationcited by 181
- 2025Memory OS of AI Agentcited by 150
- 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memorycited by 97
- 2025EgoLife: Towards Egocentric Life Assistantcited by 91
- 2019AVA-ActiveSpeaker: An Audio-Visual Dataset for Active Speaker Detectioncited by 83
- 2018Learnable PINs: Cross-Modal Embeddings for Person Identitycited by 60
- 2025WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoningcited by 53
- 2025Vgent: Graph-based Retrieval-Reasoning-Augmented Generation For Long Video Understandingcited by 31
- 2024InfiniBench: A Benchmark for Large Multi-Modal Models in Long-Form Movies and TV Showscited by 20
- 2025SiLVR: A Simple Language-based Video Reasoning Frameworkcited by 14
- 2026According to Me: Long-Term Personalized Referential Memory QAcited by 14
- 2026MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memorycited by 14
- 2024SALOVA: Segment-Augmented Long Video Assistant for Targeted Retrieval and Routing in Long-Form Video Analysiscited by 13
- 2026EgoMemReason: A Memory-Driven Reasoning Benchmark for Long-Horizon Egocentric Video Understandingcited by 13
- 2026WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interactioncited by 7
- 2025M3-SLU: Evaluating Speaker-Attributed Reasoning in Multimodal Large Language Modelscited by 4
- 2025AMUSE: Audio-Visual Benchmark and Alignment Framework for Agentic Multi-Speaker Understandingcited by 4
Built on by
Not cited yet.