mlsanityData to 7 Oct 2026

Clinical-grade Multi-Organ Pathology Report Generation for Multi-scale Whole Slide Images via a Semantically Guided Medical Text Foundation Model

arXiv 2409.15574 ↗First posted 23 Sep 2024cs.CVcited by 6

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Vision language models (VLM) have achieved success in both natural language comprehension and image recognition tasks. However, their use in pathology report generation for whole slide images (WSIs) is still limited due to the huge size of multi-scale WSIs and the high cost of WSI annotation. Moreover, in most of the existing research on pathology report generation, sufficient validation regarding clinical efficacy has not been conducted. Herein, we propose a novel Patient-level Multi-organ Pathology Report Generation (PMPRG) model, which utilizes the multi-scale WSI features from our proposed multi-scale regional vision transformer (MR-ViT) model and their real pathology reports to guide VLM training for accurate pathology report generation. The model then automatically generates a report based on the provided key features attended regional features. We assessed our model using a WSI dataset consisting of multiple organs, including the colon and kidney. Our model achieved a METEOR score of 0.68, demonstrating the effectiveness of our approach. This model allows pathologists to efficiently generate pathology reports for patients, regardless of the number of WSIs involved.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 16 of its 26 references that are in the map. First posted on arXiv 23 Sep 2024.

Builds on

16 of 26 references.

References · 16

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2020Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learningcited by 473
  4. 2022Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learningcited by 357
  5. 2017On the Automatic Generation of Medical Imaging Reportscited by 215
  6. 2021Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generationcited by 160
  7. 2022Benchmarking Self-Supervised Learning on Diverse Pathology Datasetscited by 148
  8. 2023Interactive and Explainable Region-guided Radiology Report Generationcited by 139
  9. 2023Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Imagescited by 132
  10. 2022AlignTransformer: Hierarchical Alignment of Visual Regions and Disease Tags for Medical Report Generationcited by 65
  11. 2023Text-guided Foundation Model Adaptation for Pathological Image Classificationcited by 47
  12. 2020DARE: Data Augmented Relation Extraction with GPT-2cited by 43
4 more
  1. 2022Differentiable Zooming for Multiple Instance Learning on Whole-Slide Imagescited by 43
  2. 2019Encoder-Agnostic Adaptation for Conditional Language Generationcited by 27
  3. 2023Automatic Report Generation for Histopathology images using pre-trained Vision Transformerscited by 9
  4. 2024GPC: Generative and General Pathology Image Classifiercited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Reviewcited by 9
  2. 2025Historical Report Guided Bi-modal Concurrent Learning for Pathology Report Generationcited by 7
  3. 2025PolyPath: Adapting a Large Multimodal Model for Multi-slide Pathology Report Generationcited by 4
  4. 2025HARE: an entity and relation centric evaluation framework for histopathology reportscited by 0
  5. 2026An end-to-end-trained vision-language model for native-language prostate pathology report generationcited by 0
  6. 2026Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MPath: Multimodal Pathology Report Generation from Whole Slide Images
  2. 2024PathAlign: A vision-language model for whole slide images in histopathology
  3. 2026Simple Token-Efficient Vision-Language Model for Case-level Pathology Synoptic Report Generation
  4. 2026Decomposing Whole Slide Image Report Generation with Graph-Constrained Multiple Instance Learning Workflows
  5. 2025On the Importance of Text Preprocessing for Multimodal Representation Learning and Pathology Report Generation
  6. 2025Histopathology Image Report Generation by Vision Language Model with Multimodal In-Context Learning
  7. 2024PathGen-1.6M: 1.6 Million Pathology Image-text Pairs Generation through Multi-agent Collaboration
  8. 2025Democratising Pathology Co-Pilots: An Open Pipeline and Dataset for Whole-Slide Vision-Language Modelling