mlsanityData to 14 Oct 2026

BRIT: Bidirectional Retrieval over Unified Image-Text Graph

arXiv 2505.18450 ↗First posted 24 May 2025cs.CLcited by 0

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) has emerged as a promising technique to enhance the quality and relevance of responses generated by large language models. While recent advancements have mainly focused on improving RAG for text-based queries, RAG on multi-modal documents containing both texts and images has not been fully explored. Especially when fine-tuning does not work. This paper proposes BRIT, a novel multi-modal RAG framework that effectively unifies various text-image connections in the document into a multi-modal graph and retrieves the texts and images as a query-specific sub-graph. By traversing both image-to-text and text-to-image paths in the graph, BRIT retrieve not only directly query-relevant images and texts but also further relevant contents to answering complex cross-modal multi-hop questions. To evaluate the effectiveness of BRIT, we introduce MM-RAG test set specifically designed for multi-modal question answering tasks that require to understand the text-image relations. Our comprehensive experiments demonstrate the superiority of BRIT, highlighting its ability to handle cross-modal questions on the multi-modal documents.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 12 of its 25 references that are in the map. First posted on arXiv 24 May 2025.

Builds on

12 of 25 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding7 Nov
  2. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text6 Oct

Other references · 10

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  3. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  4. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
  5. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
  6. 2022Enhancing Multi-modal and Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generationcited by 28
  7. 2023Multimodal Graph Learning for Generative Taskscited by 27
  8. 2024HOLMES: Hyper-Relational Knowledge Graphs for Multi-hop Question Answering using LLMscited by 24
  9. 2023A Suite of Generative Tasks for Multi-Level Multimodal Webpage Understandingcited by 18
  10. 2024Golden-Retriever: High-Fidelity Agentic Retrieval Augmented Generation for Industrial Knowledge Basecited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Benchmarking Retrieval-Augmented Generation in Multi-Modal Contexts
  2. 2025Cross-modal RAG: Sub-dimensional Text-to-Image Retrieval-Augmented Generation
  3. 2025Towards Mixed-Modal Retrieval for Universal Retrieval-Augmented Generation
  4. 2026Less Is More: Graph-free Multimodal RAG via Multi-signal Late Fusion
  5. 2025MHier-RAG: Multi-Modal RAG for Visual-Rich Document Question-Answering via Hierarchical and Multi-Granularity Reasoning
  6. 2025CMRAG: Co-modality-based visual document retrieval and question answering
  7. 2025MMKB-RAG: A Multi-Modal Knowledge-Based Retrieval-Augmented Generation Framework
  8. 2025Comparison of Text-Based and Image-Based Retrieval in Multimodal Retrieval Augmented Generation Large Language Model Systems