mlsanityData to 14 Oct 2026

Multimedia-Aware Question Answering: A Review of Retrieval and Cross-Modal Reasoning Architectures

arXiv 2510.20193 ↗First posted 23 Oct 2025cs.IR cs.CL cs.CVcited by 0

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Question Answering (QA) systems have traditionally relied on structured text data, but the rapid growth of multimedia content (images, audio, video, and structured metadata) has introduced new challenges and opportunities for retrieval-augmented QA. In this survey, we review recent advancements in QA systems that integrate multimedia retrieval pipelines, focusing on architectures that align vision, language, and audio modalities with user queries. We categorize approaches based on retrieval methods, fusion techniques, and answer generation strategies, and analyze benchmark datasets, evaluation protocols, and performance tradeoffs. Furthermore, we highlight key challenges such as cross-modal alignment, latency-accuracy tradeoffs, and semantic grounding, and outline open problems and future research directions for building more robust and context-aware QA systems leveraging multimedia data.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 49 of its 75 references that are in the map. First posted on arXiv 23 Oct 2025.

Builds on

49 of 75 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlook23 Mar

Other references · 48

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  7. 2015VQA: Visual Question Answeringcited by 2,506
  8. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  9. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  10. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  11. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  12. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
36 more
  1. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  2. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  3. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  4. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
  5. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  6. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 703
  7. 2018CoQA: A Conversational Question Answering Challengecited by 683
  8. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 649
  9. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 556
  10. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 517
  11. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  12. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  13. 2018TVQA: Localized, Compositional Video Question Answeringcited by 467
  14. 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
  15. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 429
  16. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 421
  17. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 413
  18. 2020Multi-modal Transformer for Video Retrievalcited by 395
  19. 2018QuAC : Question Answering in Contextcited by 381
  20. 2020HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-trainingcited by 362
  21. 2015Visual7W: Grounded Question Answering in Imagescited by 318
  22. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  23. 2018Cooperative Learning of Audio and Video Models from Self-Supervised Synchronizationcited by 249
  24. 2020Just Ask: Learning to Answer Questions from Millions of Narrated Videoscited by 248
  25. 2022Zero-Shot Video Question Answering via Frozen Bidirectional Language Modelscited by 229
  26. 2020Audio-Visual Instance Discrimination with Cross-Modal Agreementcited by 195
  27. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 191
  28. 2019TVQA+: Spatio-Temporal Grounding for Video Question Answeringcited by 154
  29. 2022The CLEAR Benchmark: Continual LEArning on Real-World Imagerycited by 93
  30. 2018Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehensioncited by 74
  31. 2023Grounded Question-Answering in Long Egocentric Videoscited by 64
  32. 2018Approximate Nearest Neighbor Search in High Dimensionscited by 47
  33. 2025Generative Retrieval and Alignment Model: A New Paradigm for E-commerce Retrievalcited by 13
  34. 2018CLEAR: A Dataset for Compositional Language and Elementary Acoustic Reasoningcited by 7
  35. 2022Evaluation of Question Answering Systems: Complexity of judging a natural languagecited by 5
  36. 2014Enhancing QA Systems with Complex Temporal Question Processing Capabilitiescited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Open-Ended and Knowledge-Intensive Video Question Answering ↗
  2. 2025RAVEN: Query-Guided Representation Alignment for Question Answering over Audio, Video, Embedded Sensors, and Natural Language
  3. 2025ReAG: Reasoning-Augmented Generation for Knowledge-based Visual Question Answering
  4. 2026WikiSeeker: Rethinking the Role of Vision-Language Models in Knowledge-Based Visual Question Answering
  5. 2025Progressive Multimodal Search and Reasoning for Knowledge-Intensive Visual Question Answering
  6. 2024Perceive, Query & Reason: Enhancing Video QA with Question-Guided Temporal Queries
  7. 2025QA-Dragon: Query-Aware Dynamic RAG System for Knowledge-Intensive Visual Question Answering
  8. 2026MARA: A Multimodal Adaptive Retrieval-Augmented Framework for Document Question Answering