mlsanityData to 14 Oct 2026

Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation

arXiv 2609.00866 ↗First posted 1 Sep 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

The rapid advancement of vision-language models (VLMs) has accelerated progress in computational pathology; however, whole-slide image (WSI)-based pathology report generation remains limited by the scarcity of large-scale WSI--report datasets and the complexity of mapping spatially distributed visual patterns to structured clinical text. To address this, we introduce a clinically curated Pan-Asia WSI--report dataset of approximately 10,500 pairs from five institutions and establish the REG 2025 benchmark through a MICCAI challenge for systematic evaluation of multimodal models. We analyze submitted methods spanning pretrained VLMs, multiple-instance learning frameworks, hierarchical expert models, retrieval-augmented generation, and cross-modal Transformers. Rather than indicating that VLM use alone was sufficient for superior performance, the results suggest that top-performing methods benefited from structured report representations, hierarchical diagnostic decomposition, and effective multimodal grounding. We identify key limitations, including instability in quantitative attribute estimation (e.g., numeric hallucination) and a tendency toward diagnostic overspecification, with some errors resembling known diagnostic pitfalls in routine pathology. These findings establish REG 2025 as a benchmark for evaluating WSI-based structured report generation and vision-language understanding in computational pathology, providing insights for the design of clinically grounded multimodal pathology models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 67 references that are in the map. First posted on arXiv 1 Sep 2026.

Builds on

28 of 67 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026HiPath: Hierarchical Vision-Language Alignment for Structured Pathology Report Prediction20 Mar
  2. 2025PathFLIP: Fine-grained Language-Image Pretraining for Versatile Computational Pathology19 Dec
  3. 2025Histopathology Image Report Generation by Vision Language Model with Multimodal In-Context Learning21 Jun
  4. 2024WSI-LLaVA: A Multimodal Large Language Model for Whole Slide Image3 Dec
  5. 2024SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understanding15 Oct
  6. 2024Clinical-grade Multi-Organ Pathology Report Generation for Multi-scale Whole Slide Images via a Semantically Guided Medical Text Foundation Model23 Sep

Other references · 22

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  3. 2015VQA: Visual Question Answeringcited by 2,506
  4. 2018Attention-based Deep Multiple Instance Learningcited by 1,095
  5. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
  6. 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 745
  7. 2021TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classificationcited by 555
  8. 2020Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learningcited by 478
  9. 2022Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learningcited by 359
  10. 2015Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classificationcited by 159
  11. 2023SlotDiffusion: Object-Centric Generative Modeling with Diffusion Modelscited by 85
  12. 2022BioBART: Pretraining and Evaluation of A Biomedical Generative Language Modelcited by 83
10 more
  1. 2025Accelerating Data Processing and Benchmarking of AI Models for Pathologycited by 65
  2. 2019BIAS: Transparent reporting of biomedical image analysis challengescited by 57
  3. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challengescited by 47
  4. 2024HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interactioncited by 43
  5. 2025HISTAI: An Open-Source, Large-Scale Whole Slide Image Dataset for Computational Pathologycited by 14
  6. 2024Aligning Knowledge Concepts to Whole Slide Images for Precise Histopathology Image Analysiscited by 10
  7. 2025Slot-MLLM: Object-Centric Visual Tokenization for Multimodal LLMcited by 7
  8. 2026QCAgent: An agentic framework for quality-controllable pathology report generation from whole slide imagecited by 4
  9. 2019Automatic Classification of Pathology Reports using TF-IDF Featurescited by 1
  10. 2025MPath: Multimodal Pathology Report Generation from Whole Slide Imagescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025PolyPath: Adapting a Large Multimodal Model for Multi-slide Pathology Report Generation
  2. 2026Simple Token-Efficient Vision-Language Model for Case-level Pathology Synoptic Report Generation
  3. 2025On the Importance of Text Preprocessing for Multimodal Representation Learning and Pathology Report Generation
  4. 2024PathAlign: A vision-language model for whole slide images in histopathology
  5. 2026PathLang: A Language-Centered Benchmark for Vision-Language Models in Computational Pathology
  6. 2026Decomposing Whole Slide Image Report Generation with Graph-Constrained Multiple Instance Learning Workflows
  7. 2026PathGLS: Evaluating Pathology Vision-Language Models without Ground Truth through Multi-Dimensional Consistency
  8. 2025Investigating Zero-Shot Diagnostic Pathology in Vision-Language Models with Efficient Prompt Design