mlsanityData to 7 Oct 2026

FunBench: Benchmarking Fundus Reading Skills of MLLMs

arXiv 2503.00901 ↗First posted 2 Mar 2025cs.CVcited by 5

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have shown significant potential in medical image analysis. However, their capabilities in interpreting fundus images, a critical skill for ophthalmology, remain under-evaluated. Existing benchmarks lack fine-grained task divisions and fail to provide modular analysis of its two key modules, i.e., large language model (LLM) and vision encoder (VE). This paper introduces FunBench, a novel visual question answering (VQA) benchmark designed to comprehensively evaluate MLLMs' fundus reading skills. FunBench features a hierarchical task organization across four levels (modality perception, anatomy perception, lesion analysis, and disease diagnosis). It also offers three targeted evaluation modes: linear-probe based VE evaluation, knowledge-prompted LLM evaluation, and holistic evaluation. Experiments on nine open-source MLLMs plus GPT-4o reveal significant deficiencies in fundus reading skills, particularly in basic tasks such as laterality recognition. The results highlight the limitations of current MLLMs and emphasize the need for domain-specific training and improved LLMs and VEs.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 23 of its 36 references that are in the map. First posted on arXiv 2 Mar 2025.

Builds on

23 of 36 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024LMOD: A Large Multimodal Ophthalmology Dataset and Benchmark for Large Vision-Language Models2 Oct
  2. 2024GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI6 Aug
  3. 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale27 Jun
  4. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM14 Feb
  5. 2023Qilin-Med-VL: Towards Chinese Large Vision-Language Model for General Healthcare27 Oct
  6. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 17

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2024GPT-4o System Cardcited by 4,467
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  6. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  7. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  8. 2022Visual Query Tuning: Towards Effective Usage of Intermediate Representations for Parameter and Memory Efficient Transfer Learningcited by 57
  9. 2018OCTID: Optical Coherence Tomography Image Databasecited by 36
  10. 2021A Benchmark of Ocular Disease Intelligent Recognition: One Shot for Multi-disease Detectioncited by 34
  11. 2019Learn to Segment Retinal Lesions and Beyondcited by 33
  12. 2023OCTDL: Optical Coherence Tomography Dataset for Image-Based Deep Learning Methodscited by 28
5 more
  1. 2024PitVQA: Image-grounded Text Embedding LLM for Visual Question Answering in Pituitary Surgerycited by 22
  2. 2020Learning Two-Stream CNN for Multi-Modal Age-related Macular Degeneration Categorizationcited by 15
  3. 2021Multi-Scale Convolutional Neural Network for Automated AMD Classification using Retinal OCT Imagescited by 10
  4. 2024Insight: A Multi-Modal Diagnostic Pipeline using LLMs for Ocular Surface Disease Diagnosiscited by 6
  5. 2024Incorporating Clinical Guidelines through Adapting Multi-modal Large Language Model for Prostate Cancer PI-RADS Scoringcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025EH-Benchmark Ophthalmic Hallucination Benchmark and Agent-Driven Top-Down Traceable Reasoning Workflowcited by 2
  2. 2025Benchmarking Large Multimodal Models for Ophthalmic Visual Question Answering with OphthalWeChatcited by 1
  3. 2026Towards Clinically Interpretable Ophthalmic VQA via Spatially-Grounded Lesion Evidencecited by 1
  4. 2026From Pixels to Explanations: Interpretable Diabetic Retinopathy Grading with CNN-Transformer Ensembles, Visual Explainability and Vision-Language Modelscited by 0
  5. 2026FOCUS: Benchmarking Retinal Model Generalization from Foundation Vision Encoders to Multimodal LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026EyeVQA: Benchmarking Ophthalmic Vision-Language Models from Recognition to Spatial Grounding
  2. 2025BYO-Eval: Build Your Own Dataset for Fine-Grained Visual Assessment of Multimodal Language Models
  3. 2025ReadBench: Measuring the Dense Text Visual Reading Ability of Vision-Language Models
  4. 2025A Novel Ophthalmic Benchmark for Evaluating Multimodal Large Language Models with Fundus Photographs and OCT Images
  5. 2024Beyond the Hype: A dispassionate look at vision-language models in medical scenario
  6. 2025OmniBrainBench: A Comprehensive Multimodal Benchmark for Brain Imaging Analysis Across Multi-stage Clinical Tasks
  7. 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision
  8. 2025Medical Large Vision Language Models with Multi-Image Visual Ability