Democratising Pathology Co-Pilots: An Open Pipeline and Dataset for Whole-Slide Vision-Language Modelling
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
Abstract (arXiv)
Vision-language models (VLMs) have the potential to become co-pilots for pathologists. However, most VLMs either focus on small regions of interest within whole-slide images, provide only static slide-level outputs, or rely on data that is not publicly available, limiting reproducibility. Furthermore, training data containing WSIs paired with detailed clinical reports is scarce, restricting progress toward transparent and generalisable VLMs. We address these limitations with three main contributions. First, we introduce Polysome, a standardised tool for synthetic instruction generation. Second, we apply Polysome to the public HISTAI dataset, generating HISTAI-Instruct, a large whole-slide instruction tuning dataset spanning 24,259 slides and over 1.1 million instruction-response pairs. Finally, we use HISTAI-Instruct to train ANTONI-α, a VLM capable of visual-question answering (VQA). We show that ANTONI-α outperforms MedGemma on WSI-level VQA tasks of tissue identification, neoplasm detection, and differential diagnosis. We also compare the performance of multiple incarnations of ANTONI-α trained with different amounts of data. All methods, data, and code are publicly available.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 12 of its 23 references that are in the map. First posted on arXiv 19 Dec 2025.
Builds on
12 of 23 references.
Closest ancestors · 3
Same direction, newest first
- 2025A Versatile Pathology Co-pilot via Reasoning Enhanced Multimodal Large Language Model23 Jul
- 2025On the Importance of Text Preprocessing for Multimodal Representation Learning and Pathology Report Generation26 Feb
- 2024SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understanding15 Oct
Other references · 9
- 2024HEST-1k: A Dataset for Spatial Transcriptomics and Histology Image Analysiscited by 104
- 2024PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathologycited by 80
- 2025Accelerating Data Processing and Benchmarking of AI Models for Pathologycited by 58
- 2024HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interactioncited by 43
- 2021SlideGraph+: Whole Slide Image Level Graphs to Predict HER2Status in Breast Cancercited by 31
- 2025Evidence-based diagnostic reasoning with multi-agent copilot for human pathologycited by 26
- 2024Multilingual Diversity Improves Vision-Language Representationscited by 14
- 2025Pathology Report Generation and Multimodal Representation Learning for Cutaneous Melanocytic Lesionscited by 6
- 2025A Multimodal Recaptioning Framework to Account for Perceptual Diversity Across Languages in Vision-Language Modelingcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation
- 2024PathGen-1.6M: 1.6 Million Pathology Image-text Pairs Generation through Multi-agent Collaboration
- 2024PathAlign: A vision-language model for whole slide images in histopathology
- 2025Investigating Zero-Shot Diagnostic Pathology in Vision-Language Models with Efficient Prompt Design
- 2026Contamination, Prior, or Evidence? Decomposing and Training Evidence Use in Whole-Slide Vision-Language Models
- 2025Evaluating Cell Type Inference in Vision Language Models Under Varying Visual Context
- 2026Automated Report-Derived Oncology VQA Benchmark for Evaluating Vision-Language Models on 3D Medical Imaging
- 2026Enhancing Pathological VLMs with Cross-scale Reasoning