mlsanityData to 7 Oct 2026

Memory-QA: Answering Recall Questions Based on Multimodal Memories

arXiv 2509.18436 ↗First posted 22 Sep 2025cs.AI cs.CL cs.DBcited by 10

Not on the map. Why?

Abstract (arXiv)

We introduce Memory-QA, a novel real-world task that involves answering recall questions about visual content from previously stored multimodal memories. This task poses unique challenges, including the creation of task-oriented memories, the effective utilization of temporal and location information within memories, and the ability to draw upon multiple memories to answer a recall question. To address these challenges, we propose a comprehensive pipeline, Pensieve, integrating memory-specific augmentation, time- and location-aware multi-signal retrieval, and multi-memory QA fine-tuning. We created a multimodal benchmark to illustrate various real challenges in this task, and show the superior performance of Pensieve over state-of-the-art solutions (up to 14% on QA accuracy).

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 25 of its 47 references that are in the map. First posted on arXiv 22 Sep 2025.

Builds on

25 of 47 references.

References · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  5. 2015VQA: Visual Question Answeringcited by 2,489
  6. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  7. 2021An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQAcited by 298
  8. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 237
  9. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Textcited by 216
  10. 2021MultiModalQA: Complex Question Answering over Text, Tables and Imagescited by 177
  11. 2023BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questionscited by 125
  12. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 116
13 more
  1. 2024Unifying Multimodal Retrieval via Document Screenshot Embeddingcited by 107
  2. 2022Retrieval Augmented Visual Question Answering with Outside Knowledgecited by 93
  3. 2024VISTA: Visualized Text Embedding For Universal Multi-Modal Retrievalcited by 67
  4. 2023Learning Customized Visual Models with Retrieval-Augmented Knowledgecited by 58
  5. 2024MLLM Is a Strong Reranker: Advancing Multimodal Retrieval-augmented Generation via Knowledge-enhanced Reranking and Noise-injected Trainingcited by 50
  6. 2020ManyModalQA: Modality Disambiguation and QA over Diverse Inputscited by 38
  7. 2025Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlookcited by 32
  8. 2022Enhancing Multi-modal and Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generationcited by 28
  9. 2023Unified Language Representation for Question Answering over Text, Tables, and Imagescited by 22
  10. 2023Fast and Accurate Factual Inconsistency Detection Over Long Documentscited by 22
  11. 2024SnapNTell: Enhancing Entity-Centric Visual Question Answering with Retrieval Augmented Multimodal LLMcited by 18
  12. 2024Retrieval Meets Reasoning: Even High-school Textbook Knowledge Benefits Multimodal Reasoningcited by 13
  13. 2025Retrieval-Augmented Visual Question Answering via Built-in Autoregressive Search Enginescited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 9.

10 papers.

  1. 2026STALE: Can LLM Agents Know When Their Memories Are No Longer Valid?cited by 36
  2. 2025AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agentscited by 12
  3. 2026According to Me: Long-Term Personalized Referential Memory QAcited by 12
  4. 2026EpiBench: Benchmarking Multi-turn Research Workflows for Multimodal Agentscited by 2
  5. 2026Enhancing Conversational Agents via Task-Oriented Adversarial Memory Adaptationcited by 1
  6. 2026SaliMory: Orchestrating Cognitive Memory for Conversational Agentscited by 1
  7. 2026CMI-Mem: Toward Generalizable Long-Term Memory Management via CMI-Augmented Reinforcement Learningcited by 1
  8. 2026GraphMemix: Query-Aware Evidence Forests for Long-Term Multimodal Agent Memorycited by 1
  9. 2026Life-Bench: A Benchmark and Knowledge Graph Framework for Multimodal Personalization Beyond Concept Recognitioncited by 0
  10. 2026When Helpful Text Hurts: Option-Redirecting Bias in Vision-Language Modelscited by 0