# WsiCaption: Multiple Instance Generation of Pathology Reports for Gigapixel Whole-Slide Images

arXiv: https://arxiv.org/abs/2311.16480 · HTML: https://mlsanity.com/paper/2311.16480/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 27 Nov 2023.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [Self-critical Sequence Training for Image Captioning](https://mlsanity.com/paper/1612.00563/) (2016)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Generating Radiology Reports via Memory-driven Transformer](https://mlsanity.com/paper/2010.16056/) (2020)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification](https://mlsanity.com/paper/2203.12081/) (2022)
- [On the Automatic Generation of Medical Imaging Reports](https://mlsanity.com/paper/1711.08195/) (2017)
- [Benchmarking Self-Supervised Learning on Diverse Pathology Datasets](https://mlsanity.com/paper/2212.04690/) (2022)
- [Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation](https://mlsanity.com/paper/2010.10042/) (2020)
- [Clinically Accurate Chest X-Ray Report Generation](https://mlsanity.com/paper/1904.02633/) (2019)
- [Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Images](https://mlsanity.com/paper/2306.07831/) (2023)
- [Multiple Instance Captioning: Learning Representations from Histopathology Textbooks and Articles](https://mlsanity.com/paper/2103.05121/) (2021)
- [Exploring Unsupervised Cell Recognition with Prior Self-activation Maps](https://mlsanity.com/paper/2308.11144/) (2023)

## Built on by (most cited first, 30 of 31)

- [HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interaction](https://mlsanity.com/paper/2403.05396/) (2024)
- [WSI-LLaVA: A Multimodal Large Language Model for Whole Slide Image](https://mlsanity.com/paper/2412.02141/) (2024)
- [CPath-Omni: A Unified Multimodal Foundation Model for Patch and Whole Slide Image Analysis in Computational Pathology](https://mlsanity.com/paper/2412.12077/) (2024)
- [Historical Report Guided Bi-modal Concurrent Learning for Pathology Report Generation](https://mlsanity.com/paper/2506.18658/) (2025)
- [PathNavigate: A Training-Free Pathology Agent with Surprise-Guided Scan and Shared Slide Memory for Whole-Slide Image VQA](https://mlsanity.com/paper/2605.23559/) (2026)
- [Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challenges](https://mlsanity.com/paper/2505.10993/) (2025)
- [Act Like a Pathologist: Tissue-Aware Whole Slide Image Reasoning](https://mlsanity.com/paper/2603.00667/) (2026)
- [PolyPath: Adapting a Large Multimodal Model for Multi-slide Pathology Report Generation](https://mlsanity.com/paper/2502.10536/) (2025)
- [QCAgent: An agentic framework for quality-controllable pathology report generation from whole slide image](https://mlsanity.com/paper/2603.01647/) (2026)
- [LAMMI-Pathology: A Tool-Centric Bottom-Up LVLM-Agent Framework for Molecularly Informed Medical Intelligence in Pathology](https://mlsanity.com/paper/2602.18773/) (2026)
- [Designing UNICORN: a Unified Benchmark for Imaging in Computational Pathology, Radiology, and Natural Language](https://mlsanity.com/paper/2603.02790/) (2026)
- [RANGER: Sparsely-Gated Mixture-of-Experts with Adaptive Retrieval Re-ranking for Pathology Report Generation](https://mlsanity.com/paper/2603.04348/) (2026)
- [NuNext: Reframing Nucleus Detection as Next-Point Detection](https://mlsanity.com/paper/2603.07098/) (2026)
- [Learning Spatial-Preserving Hierarchical Representations for Digital Pathology](https://mlsanity.com/paper/2406.09333/) (2024)
- [CLIP-IT: CLIP-based Pairing for Histology Images Classification](https://mlsanity.com/paper/2504.16181/) (2025)
- [Multi-View Stenosis Classification Leveraging Transformer-Based Multiple-Instance Learning Using Real-World Clinical Data](https://mlsanity.com/paper/2602.02067/) (2026)
- [PathMem: Toward Cognition-Aligned Memory Transformation for Pathology MLLMs](https://mlsanity.com/paper/2603.09943/) (2026)
- [Semantic Context-aware mOdality fUsion Transformer (SCOUT): A Context-Aware Multimodal Transformer for Concept-Grounded Pathology Report Generation](https://mlsanity.com/paper/2605.01144/) (2026)
- [Auditing Data Leakage in Whole-Slide Image Multimodal Benchmarks](https://mlsanity.com/paper/2607.12278/) (2026)
- [Any-to-Any Learning in Computational Pathology via Triplet Multimodal Pretraining](https://mlsanity.com/paper/2505.12711/) (2025)
- [PA-MIL: Phenotype-Aware Multiple Instance Learning Guided by Language Prompting and Genotype-to-Phenotype Relationships](https://mlsanity.com/paper/2602.02558/) (2026)
- [Footprint-Guided Exemplar-Free Continual Histopathology Report Generation](https://mlsanity.com/paper/2602.23817/) (2026)
- [Layout-Guided Controllable Pathology Image Generation with In-Context Diffusion Transformers](https://mlsanity.com/paper/2603.13386/) (2026)
- [PBSBench: A Multi-Level Vision-Language Framework and Benchmark for Hematopathology Whole Slide Image Interpretation](https://mlsanity.com/paper/2604.17570/) (2026)
- [Prior-Anchored Debiasing for Long-Tailed Multi-Organ Pathology Report Generation](https://mlsanity.com/paper/2607.00499/) (2026)
- [LaGuadia: Language-Guided Adaptive Distillation from Pathology Foundation Models](https://mlsanity.com/paper/2607.11257/) (2026)
- [PathReportEval: A Systematic Benchmark for Pathology Report Generation](https://mlsanity.com/paper/2607.18448/) (2026)
- [Pathologist Attention-Aligned Report Generation for Prostate Histopathology](https://mlsanity.com/paper/2607.19624/) (2026)
- [From Multi-Resolution Cells to Gigapixel Whole Slide Images Foundation Model for Computational Pathology](https://mlsanity.com/paper/2608.03508/) (2026)
- [Decomposing Whole Slide Image Report Generation with Graph-Constrained Multiple Instance Learning Workflows](https://mlsanity.com/paper/2608.15353/) (2026)

## Abstract (arXiv)

Whole slide images are the foundation of digital pathology for the diagnosis and treatment of carcinomas. Writing pathology reports is laborious and error-prone for inexperienced pathologists. To reduce the workload and improve clinical automation, we investigate how to generate pathology reports given whole slide images. On the data end, we curated the largest WSI-text dataset (PathText). In specific, we collected nearly 10000 high-quality WSI-text pairs for visual-language models by recognizing and cleaning pathology reports which narrate diagnostic slides in TCGA. On the model end, we propose the multiple instance generative model (MI-Gen) which can produce pathology reports for gigapixel WSIs. We benchmark our model on the largest subset of TCGA-PathoText. Experimental results show our model can generate pathology reports which contain multiple clinical clues and achieve competitive performance on certain slide-level tasks. We observe that simple semantic extraction from the pathology reports can achieve the best performance (0.838 of F1 score) on BRCA subtyping surpassing previous state-of-the-art approaches. Our collected dataset and related code are available.