mlsanityData to 14 Oct 2026

An end-to-end-trained vision-language model for native-language prostate pathology report generation

arXiv 2608.23143 ↗First posted 24 Aug 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Prostate cancer is among the most frequently diagnosed malignancies worldwide, and structured reporting of each biopsy core burdens pathologists. Existing tools frame this as classification, leaving pathologists to assemble coherent reports, while many slide-level vision-language models rely on English-centric encoders that transfer poorly to other clinical languages. We present a slide-level framework generating prostate biopsy reports that is language-independent by construction: tokenizer and model are trained from scratch, demonstrated here in German. To address paired-data scarcity, an automated pipeline uses a locally deployed large language model to split composite reports into core-specific image-text pairs, yielding 17,344 pairs from 2,402 historical cases without manual annotation. Evaluated for clinical attributes rather than linguistic similarity, the model achieves 96.2% F1 for malignancy detection and 65.2% for Gleason grading, competitive with an FDA-cleared classifier. Grading is further validated on three external cohorts with latent-space augmentation. Institutions can thus train native-language reporting models on their own archives.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 23 of its 51 references that are in the map. First posted on arXiv 24 Aug 2026.

Builds on

23 of 51 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Clinical-grade Multi-Organ Pathology Report Generation for Multi-scale Whole Slide Images via a Semantically Guided Medical Text Foundation Model23 Sep
  2. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 21

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  5. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  6. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  7. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  8. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  9. 2018Hierarchical Neural Story Generationcited by 1,193
  10. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  11. 2018Attention-based Deep Multiple Instance Learningcited by 1,095
  12. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
9 more
  1. 2024Multimodal Whole Slide Foundation Model for Pathologycited by 162
  2. 2019What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Usecited by 135
  3. 2024PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathologycited by 85
  4. 2025Accelerating Data Processing and Benchmarking of AI Models for Pathologycited by 65
  5. 2026MedGemma 1.5 Technical Reportcited by 45
  6. 2025Current Pathology Foundation Models are unrobust to Medical Center Differencescited by 39
  7. 2026Scanner-Induced Domain Shifts Undermine the Robustness of Pathology Foundation Modelscited by 9
  8. 2024Large Language Models-Enabled Digital Twins for Precision Medicine in Rare Gynecological Tumorscited by 2
  9. 2025Controllable Latent Space Augmentation for Digital Pathologycited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation
  2. 2026Simple Token-Efficient Vision-Language Model for Case-level Pathology Synoptic Report Generation
  3. 2026A Unified Vision-Language Model for PSMA PET/CT Report Generation, Visual Question Answering, and Lesion Segmentation
  4. 2025On the Importance of Text Preprocessing for Multimodal Representation Learning and Pathology Report Generation
  5. 2024PathAlign: A vision-language model for whole slide images in histopathology
  6. 2025Democratising Pathology Co-Pilots: An Open Pipeline and Dataset for Whole-Slide Vision-Language Modelling
  7. 2025MPath: Multimodal Pathology Report Generation from Whole Slide Images
  8. 2025Histopathology Image Report Generation by Vision Language Model with Multimodal In-Context Learning