# Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation

arXiv: https://arxiv.org/abs/2609.00866 · HTML: https://mlsanity.com/paper/2609.00866/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 28 of its 67 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Medical vision-language models](https://mlsanity.com/mllm/medical-vision-language-models/)

## Builds on (most cited first, 28 of 28)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Multimodal Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2302.00923/) (2023)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2106.00908/) (2021)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification](https://mlsanity.com/paper/1504.07947/) (2015)
- [SlotDiffusion: Object-Centric Generative Modeling with Diffusion Models](https://mlsanity.com/paper/2305.11281/) (2023)
- [BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model](https://mlsanity.com/paper/2204.03905/) (2022)
- [Accelerating Data Processing and Benchmarking of AI Models for Pathology](https://mlsanity.com/paper/2502.06750/) (2025)
- [BIAS: Transparent reporting of biomedical image analysis challenges](https://mlsanity.com/paper/1910.04071/) (2019)
- [SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understanding](https://mlsanity.com/paper/2410.11761/) (2024)
- [A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges](https://mlsanity.com/paper/2501.02189/) (2025)
- [HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interaction](https://mlsanity.com/paper/2403.05396/) (2024)
- [WSI-LLaVA: A Multimodal Large Language Model for Whole Slide Image](https://mlsanity.com/paper/2412.02141/) (2024)
- [HISTAI: An Open-Source, Large-Scale Whole Slide Image Dataset for Computational Pathology](https://mlsanity.com/paper/2505.12120/) (2025)
- [Aligning Knowledge Concepts to Whole Slide Images for Precise Histopathology Image Analysis](https://mlsanity.com/paper/2411.18101/) (2024)
- [Slot-MLLM: Object-Centric Visual Tokenization for Multimodal LLM](https://mlsanity.com/paper/2505.17726/) (2025)
- [Clinical-grade Multi-Organ Pathology Report Generation for Multi-scale Whole Slide Images via a Semantically Guided Medical Text Foundation Model](https://mlsanity.com/paper/2409.15574/) (2024)
- [PathFLIP: Fine-grained Language-Image Pretraining for Versatile Computational Pathology](https://mlsanity.com/paper/2512.17621/) (2025)
- [QCAgent: An agentic framework for quality-controllable pathology report generation from whole slide image](https://mlsanity.com/paper/2603.01647/) (2026)
- [Histopathology Image Report Generation by Vision Language Model with Multimodal In-Context Learning](https://mlsanity.com/paper/2506.17645/) (2025)
- [Automatic Classification of Pathology Reports using TF-IDF Features](https://mlsanity.com/paper/1903.07406/) (2019)
- [MPath: Multimodal Pathology Report Generation from Whole Slide Images](https://mlsanity.com/paper/2512.11906/) (2025)
- [HiPath: Hierarchical Vision-Language Alignment for Structured Pathology Report Prediction](https://mlsanity.com/paper/2603.19957/) (2026)

## Abstract (arXiv)

The rapid advancement of vision-language models (VLMs) has accelerated progress in computational pathology; however, whole-slide image (WSI)-based pathology report generation remains limited by the scarcity of large-scale WSI--report datasets and the complexity of mapping spatially distributed visual patterns to structured clinical text. To address this, we introduce a clinically curated Pan-Asia WSI--report dataset of approximately 10,500 pairs from five institutions and establish the REG 2025 benchmark through a MICCAI challenge for systematic evaluation of multimodal models. We analyze submitted methods spanning pretrained VLMs, multiple-instance learning frameworks, hierarchical expert models, retrieval-augmented generation, and cross-modal Transformers. Rather than indicating that VLM use alone was sufficient for superior performance, the results suggest that top-performing methods benefited from structured report representations, hierarchical diagnostic decomposition, and effective multimodal grounding. We identify key limitations, including instability in quantitative attribute estimation (e.g., numeric hallucination) and a tendency toward diagnostic overspecification, with some errors resembling known diagnostic pitfalls in routine pathology. These findings establish REG 2025 as a benchmark for evaluating WSI-based structured report generation and vision-language understanding in computational pathology, providing insights for the design of clinically grounded multimodal pathology models.