mlsanityData to 7 Oct 2026

Towards Retrieval Augmented Generation over Large Video Libraries

arXiv 2406.14938 ↗First posted 21 Jun 2024cs.CL cs.AIcited by 4

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Video content creators need efficient tools to repurpose content, a task that often requires complex manual or automated searches. Crafting a new video from large video libraries remains a challenge. In this paper we introduce the task of Video Library Question Answering (VLQA) through an interoperable architecture that applies Retrieval Augmented Generation (RAG) to video libraries. We propose a system that uses large language models (LLMs) to generate search queries, retrieving relevant video moments indexed by speech and visual metadata. An answer generation module then integrates user queries with this metadata to produce responses with specific video timestamps. This approach shows promise in multimedia content retrieval, and AI-assisted video content creation.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 16 of its 23 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

16 of 23 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024iRAG: Advancing RAG for Videos with an Incremental Approach18 Apr

Other references · 15

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  5. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  6. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
  7. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  8. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
  9. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
  10. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 326
  11. 2024CBR-RAG: Case-Based Reasoning for Retrieval Augmented Generation in LLMs for Legal Question Answeringcited by 70
  12. 2023DiffusionRet: Generative Text-Video Retrieval with Diffusion Modelcited by 52
3 more
  1. 2023Unified Coarse-to-Fine Alignment for Video-Text Retrievalcited by 49
  2. 2016Learning Joint Representations of Videos and Sentences with Web Image Searchcited by 17
  3. 2023Deep Learning for Video-Text Retrieval: a Reviewcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026Demo-ICL: In-Context Learning for Procedural Video Knowledge Acquisitioncited by 8
  2. 2025Atom: Efficient On-Device Video-Language Pipelines Through Modular Reusecited by 0
  3. 2026MultiVENT-Raw: A Benchmark for Retrieval and Reasoning over Raw Videoscited by 0
  4. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025VideoRAG: Retrieval-Augmented Generation over Video Corpus
  2. 2025Retrieval-Augmented Visual Question Answering via Built-in Autoregressive Search Engines
  3. 2024Searching for Best Practices in Retrieval-Augmented Generation
  4. 2025VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videos
  5. 2024Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehension
  6. 2024Video Enriched Retrieval Augmented Generation Using Aligned Video Captions
  7. 2025E-VRAG: Enhancing Long Video Understanding with Resource-Efficient Retrieval Augmented Generation
  8. 2026Rethinking RAG in Long Videos: What to Retrieve and How to Use It?