mlsanityData to 7 Oct 2026

Memory Helps, but Confabulation Misleads: Understanding Streaming Events in Videos with MLLMs

arXiv 2502.15457 ↗First posted 21 Feb 2025cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Multimodal large language models (MLLMs) have demonstrated strong performance in understanding videos holistically, yet their ability to process streaming videos-videos are treated as a sequence of visual events-remains underexplored. Intuitively, leveraging past events as memory can enrich contextual and temporal understanding of the current event. In this paper, we show that leveraging memories as contexts helps MLLMs better understand video events. However, because such memories rely on predictions of preceding events, they may contain misinformation, leading to confabulation and degraded performance. To address this, we propose a confabulation-aware memory modification method that mitigates confabulated memory for memory-enhanced event understanding.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 33 of its 39 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

33 of 39 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding8 Apr
  2. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding18 Mar
  3. 2023Eliciting In-Context Learning in Vision-Language Models for Videos Through Curated Data Distributional Properties28 Nov
  4. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding5 Jun

Other references · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  11. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  12. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
17 more
  1. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  2. 2022Emergent Abilities of Large Language Modelscited by 2,483
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  4. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  5. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  6. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  7. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  8. 2024Titans: Learning to Memorize at Test Timecited by 303
  9. 2024Streaming Long Video Understanding with Large Language Modelscited by 164
  10. 2024Human-inspired Episodic Memory for Infinite Context LLMscited by 62
  11. 2024Larimar: Large Language Models with Episodic Memory Controlcited by 53
  12. 2023MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learningcited by 46
  13. 2024Confabulation: The Surprising Value of Large Language Model Hallucinationscited by 42
  14. 2023TextBind: Multi-turn Interleaved Multimodal Instruction-following in the Wildcited by 22
  15. 2024CrAM: Credibility-Aware Attention Modification in LLMs for Combating Misinformation in RAGcited by 22
  16. 2024COSMO: COntrastive Streamlined MultimOdal Model with Interleaved Pre-Trainingcited by 18
  17. 2024CaMML: Context-Aware Multimodal Learner for Large Modelscited by 6

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmarkcited by 37
  2. 2025Machine Mirages: Defining the Undefinedcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events
  2. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMs
  3. 2024Temporal Insight Enhancement: Mitigating Temporal Hallucination in Multimodal Large Language Models
  4. 2024Towards Event-oriented Long Video Understanding
  5. 2026Towards a Dynamic and Fixed-budget Memory Bank for Efficient Streaming Video Understanding
  6. 2025What Happens When: Learning Temporal Orders of Events in Videos
  7. 2026Scaling the Long Video Understanding of Multimodal Large Language Models via Visual Memory Mechanism
  8. 2026FlashBack: Knowing When to Remember in Streaming Vision-Language Models