mlsanityData to 14 Oct 2026

VLR-Bench: Multilingual Benchmark Dataset for Vision-Language Retrieval Augmented Generation

arXiv 2412.10151 ↗First posted 13 Dec 2024cs.CV cs.AI cs.CLcited by 3

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

also in 1 more
Abstract (arXiv)

We propose the VLR-Bench, a visual question answering (VQA) benchmark for evaluating vision language models (VLMs) based on retrieval augmented generation (RAG). Unlike existing evaluation datasets for external knowledge-based VQA, the proposed VLR-Bench includes five input passages. This allows testing of the ability to determine which passage is useful for answering a given query, a capability lacking in previous research. In this context, we constructed a dataset of 32,000 automatically generated instruction-following examples, which we denote as VLR-IF. This dataset is specifically designed to enhance the RAG capabilities of VLMs by enabling them to learn how to generate appropriate answers based on input passages. We evaluated the validity of the proposed benchmark and training data and verified its performance using the state-of-the-art Llama3-based VLM, the Llava-Llama-3 model. The proposed VLR-Bench and VLR-IF datasets are publicly available online.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 23 references that are in the map. First posted on arXiv 13 Dec 2024.

Builds on

14 of 23 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond24 Aug

Other references · 13

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  4. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  5. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  6. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 207
  7. 2023Encyclopedic VQA: Visual questions about detailed properties of fine-grained categoriescited by 107
  8. 2022Retrieval Augmented Visual Question Answering with Outside Knowledgecited by 94
  9. 2021xGQA: Cross-Lingual Visual Question Answeringcited by 72
  10. 2021Weakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answeringcited by 50
  11. 2021Select, Substitute, Search: A New Benchmark for Knowledge-Augmented Visual Question Answeringcited by 33
  12. 2024X-LLaVA: Optimizing Bilingual Large Vision-Language Alignmentcited by 10
1 more
  1. 2024BOK-VQA: Bilingual outside Knowledge-Based Visual Question Answering via Graph Representation Pretrainingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoningcited by 14
  2. 2025KOFFVQA: An Objectively Evaluated Free-form VQA Benchmark for Large Vision-Language Models in the Korean Languagecited by 5
  3. 2026Pix2Fact: When Vision Is Not Enough -- Benchmarking Fine-Grained VQA with Web Verification on High-Resolution Real-World Scenescited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2024Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy
  2. 2024Guiding Vision-Language Model Selection for Visual Question-Answering Across Tasks, Domains, and Knowledge Types
  3. 2026WikiSeeker: Rethinking the Role of Vision-Language Models in Knowledge-Based Visual Question Answering
  4. 2025M4-RAG: A Massive-Scale Multilingual Multi-Cultural Multimodal RAG
  5. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark
  6. 2024Towards Flexible Evaluation for Generative Visual Question Answering
  7. 2024AutoBench-V: Can Large Vision-Language Models Benchmark Themselves?
  8. 2025Detecting Knowledge Boundary of Vision Large Language Models by Sampling-Based Inference