mlsanityData to 7 Oct 2026

MARQUIS: A Three-Stage Pipeline for Video Retrieval-Augmented Generation

arXiv 2605.17640 ↗First posted 17 May 2026cs.IR cs.CVcited by 0

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Retrieval-augmented generation from videos requires systems to retrieve relevant audiovisual evidence from large corpora and synthesize it into coherent, attributed text. Current approaches struggle at both ends: retrieval methods fail on complex, multi-faceted queries that cannot be captured by a single embedding, while generation methods lack the high-level reasoning needed to synthesize across multiple videos and face memory constraints over long, multi-video contexts. We present MARQUIS: a three-stage pipeline that addresses these limitations through (1) query expansion, fusion, and reranking, (2) calibrated structured evidence extraction, and (3) article generation from extracted evidence, optionally controlled by an RLM. On the MAGMaR2026 shared task, we improve retrieval performance from 0.195 to 0.759 (nDCG@10). For article generation, ITER-QA-BASE improves average human score from 3.09 to 3.83 over the CAG baseline, while MARQUIS-RLM achieves a human score of 3.30 and the strongest citation recall among non-QA systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 22 of its 58 references that are in the map. First posted on arXiv 17 May 2026.

Builds on

22 of 58 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025WikiVideo: Article Generation from Multiple Videos1 Apr

Other references · 21

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  4. 2017Dense-Captioning Events in Videoscited by 789
  5. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
  6. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 559
  7. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  8. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 366
  9. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 303
  10. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 223
  11. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 212
  12. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 211
9 more
  1. 2024FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructionscited by 67
  2. 2025Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Taskscited by 62
  3. 2025Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agentscited by 37
  4. 2025Tevatron 2.0: Unified Document Retrieval Toolkit across Scale, Language, and Modalitycited by 30
  5. 2025Video-ColBERT: Contextualized Late Interaction for Text-to-Video Retrievalcited by 20
  6. 2024MultiVENT 2.0: A Massive Multilingual Benchmark for Event-Centric Video Retrievalcited by 17
  7. 2025GINGER: Grounded Information Nugget-Based Generation of Responsescited by 16
  8. 2025MMMORRF: Multimodal Multilingual Modularized Reciprocal Rank Fusioncited by 8
  9. 2026Multi-Vector Index Compression in Any Modalitycited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026PREGEN: Uncovering Latent Thoughts in Composed Video Retrieval
  2. 2026Decoupling Semantics and Logic: A Training-Free Coarse-to-Fine Pipeline for Video Retrieval-Augmented Generation
  3. 2026Rethinking RAG in Long Videos: What to Retrieve and How to Use It?
  4. 2025VideoRAG: Retrieval-Augmented Generation over Video Corpus
  5. 2025VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videos
  6. 2026R^3: Composed Video Retrieval via Reasoning-Guided Recalling and Re-ranking
  7. 2024Towards Retrieval Augmented Generation over Large Video Libraries
  8. 2025CFVBench: A Comprehensive Video Benchmark for Fine-grained Multimodal Retrieval-Augmented Generation