mlsanityData to 13 Oct 2026

Self-adaptive Multimodal Retrieval-Augmented Generation

arXiv 2410.11321 ↗First posted 15 Oct 2024cs.CLcited by 4

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Traditional Retrieval-Augmented Generation (RAG) methods are limited by their reliance on a fixed number of retrieved documents, often resulting in incomplete or noisy information that undermines task performance. Although recent adaptive approaches alleviated these problems, their application in intricate and real-world multimodal tasks remains limited. To address these, we propose a new approach called Self-adaptive Multimodal Retrieval-Augmented Generation (SAM-RAG), tailored specifically for multimodal contexts. SAM-RAG not only dynamically filters relevant documents based on the input query, including image captions when needed, but also verifies the quality of both the retrieved documents and the output. Extensive experimental results show that SAM-RAG surpasses existing state-of-the-art methods in both retrieval accuracy and response generation. By further ablation experiments and effectiveness analysis, SAM-RAG maintains high recall quality while improving overall task performance in multimodal RAG task. Our codes are available at https://github.com/SAM-RAG/SAM_RAG.

Key facts. Cited by 4 papers in the map (data through 13 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 29 of its 45 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

29 of 45 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Retrieval Augmented Visual Question Answering with Outside Knowledge7 Oct
  2. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text6 Oct

Other references · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,856
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,221
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,492
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,657
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,133
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,797
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,367
  9. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,815
  10. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,450
  11. 2022Large Language Models Can Self-Improvecited by 659
  12. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 353
15 more
  1. 2022Generating Sequences by Learning to Self-Correctcited by 273
  2. 2021MultiModalQA: Complex Question Answering over Text, Tables and Imagescited by 177
  3. 2022A Survey on Retrieval-Augmented Text Generationcited by 158
  4. 2022Re2G: Retrieve, Rerank, Generatecited by 102
  5. 2024Efficient Multimodal Large Language Models: A Surveycited by 95
  6. 2024CodeRAG-Bench: Can Retrieval Augment Code Generation?cited by 91
  7. 2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 85
  8. 2024Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Basescited by 77
  9. 2023UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluationcited by 75
  10. 2023Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM Familycited by 71
  11. 2024Retrieval-Augmented Generation for Natural Language Processing: A Surveycited by 71
  12. 2024Improving Retrieval Augmented Language Model with Self-Reasoningcited by 28
  13. 2019Learning Alignment for Multimodal Emotion Recognition from Speechcited by 22
  14. 2023MMHQA-ICL: Multimodal In-context Learning for Hybrid Question Answering over Text, Tables and Imagescited by 18
  15. 2024Optimizing Query Generation for Enhanced Document Retrieval in RAGcited by 10

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directionscited by 30
  2. 2025ViDove: A Translation Agent System with Multimodal Context and Memory-Augmented Reasoningcited by 3
  3. 2025Empowering Multimodal LLMs with External Tools: A Comprehensive Surveycited by 2
  4. 2026Knowledge Is Not Static: Order-Aware Hypergraph RAG for Language Modelscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2024Searching for Best Practices in Retrieval-Augmented Generation
  2. 2026Evaluating Modern RAG: Textual, Multimodal, Dense, and Late Interaction Pipelines
  3. 2025Re-ranking the Context for Multimodal Retrieval Augmented Generation
  4. 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation
  5. 2025RAG-Check: Evaluating Multimodal Retrieval Augmented Generation Performance
  6. 2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions
  7. 2025Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlook
  8. 2025Are We on the Right Way for Assessing Document Retrieval-Augmented Generation?