mlsanityData to 14 Oct 2026

WsiCaption: Multiple Instance Generation of Pathology Reports for Gigapixel Whole-Slide Images

arXiv 2311.16480 ↗First posted 27 Nov 2023cs.CV cs.AI cs.CLcited by 31

Not on the map. Why?

Abstract (arXiv)

Whole slide images are the foundation of digital pathology for the diagnosis and treatment of carcinomas. Writing pathology reports is laborious and error-prone for inexperienced pathologists. To reduce the workload and improve clinical automation, we investigate how to generate pathology reports given whole slide images. On the data end, we curated the largest WSI-text dataset (PathText). In specific, we collected nearly 10000 high-quality WSI-text pairs for visual-language models by recognizing and cleaning pathology reports which narrate diagnostic slides in TCGA. On the model end, we propose the multiple instance generative model (MI-Gen) which can produce pathology reports for gigapixel WSIs. We benchmark our model on the largest subset of TCGA-PathoText. Experimental results show our model can generate pathology reports which contain multiple clinical clues and achieve competitive performance on certain slide-level tasks. We observe that simple semantic extraction from the pathology reports can achieve the best performance (0.838 of F1 score) on BRCA subtyping surpassing previous state-of-the-art approaches. Our collected dataset and related code are available.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Builds on 20 of its 29 references that are in the map. First posted on arXiv 27 Nov 2023.

Builds on

20 of 29 references.

References · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  6. 2018Attention-based Deep Multiple Instance Learningcited by 1,095
  7. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  8. 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 745
  9. 2016Self-critical Sequence Training for Image Captioningcited by 547
  10. 2020Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learningcited by 478
  11. 2020Generating Radiology Reports via Memory-driven Transformercited by 368
  12. 2022Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learningcited by 359
8 more
  1. 2022DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classificationcited by 256
  2. 2017On the Automatic Generation of Medical Imaging Reportscited by 216
  3. 2022Benchmarking Self-Supervised Learning on Diverse Pathology Datasetscited by 150
  4. 2020Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generationcited by 141
  5. 2019Clinically Accurate Chest X-Ray Report Generationcited by 139
  6. 2023Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Imagescited by 134
  7. 2021Multiple Instance Captioning: Learning Representations from Histopathology Textbooks and Articlescited by 58
  8. 2023Exploring Unsupervised Cell Recognition with Prior Self-activation Mapscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 5; 2026: 22.

31 papers.

Sort:

By area:

  1. 2024HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interactioncited by 43
  2. 2024WSI-LLaVA: A Multimodal Large Language Model for Whole Slide Imagecited by 40
  3. 2024CPath-Omni: A Unified Multimodal Foundation Model for Patch and Whole Slide Image Analysis in Computational Pathologycited by 34
  4. 2025Historical Report Guided Bi-modal Concurrent Learning for Pathology Report Generationcited by 7
  5. 2026PathNavigate: A Training-Free Pathology Agent with Surprise-Guided Scan and Shared Slide Memory for Whole-Slide Image VQAcited by 7
  6. 2025Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challengescited by 5
  7. 2026Act Like a Pathologist: Tissue-Aware Whole Slide Image Reasoningcited by 5
  8. 2025PolyPath: Adapting a Large Multimodal Model for Multi-slide Pathology Report Generationcited by 4
  9. 2026QCAgent: An agentic framework for quality-controllable pathology report generation from whole slide imagecited by 4
  10. 2026LAMMI-Pathology: A Tool-Centric Bottom-Up LVLM-Agent Framework for Molecularly Informed Medical Intelligence in Pathologycited by 3
  11. 2026Designing UNICORN: a Unified Benchmark for Imaging in Computational Pathology, Radiology, and Natural Languagecited by 3
  12. 2026RANGER: Sparsely-Gated Mixture-of-Experts with Adaptive Retrieval Re-ranking for Pathology Report Generationcited by 3
19 more
  1. 2026NuNext: Reframing Nucleus Detection as Next-Point Detectioncited by 2
  2. 2024Learning Spatial-Preserving Hierarchical Representations for Digital Pathologycited by 1
  3. 2025CLIP-IT: CLIP-based Pairing for Histology Images Classificationcited by 1
  4. 2026Multi-View Stenosis Classification Leveraging Transformer-Based Multiple-Instance Learning Using Real-World Clinical Datacited by 1
  5. 2026PathMem: Toward Cognition-Aligned Memory Transformation for Pathology MLLMscited by 1
  6. 2026Semantic Context-aware mOdality fUsion Transformer (SCOUT): A Context-Aware Multimodal Transformer for Concept-Grounded Pathology Report Generationcited by 1
  7. 2026Auditing Data Leakage in Whole-Slide Image Multimodal Benchmarkscited by 1
  8. 2025Any-to-Any Learning in Computational Pathology via Triplet Multimodal Pretrainingcited by 0
  9. 2026PA-MIL: Phenotype-Aware Multiple Instance Learning Guided by Language Prompting and Genotype-to-Phenotype Relationshipscited by 0
  10. 2026Footprint-Guided Exemplar-Free Continual Histopathology Report Generationcited by 0
  11. 2026Layout-Guided Controllable Pathology Image Generation with In-Context Diffusion Transformerscited by 0
  12. 2026PBSBench: A Multi-Level Vision-Language Framework and Benchmark for Hematopathology Whole Slide Image Interpretationcited by 0
  13. 2026Prior-Anchored Debiasing for Long-Tailed Multi-Organ Pathology Report Generationcited by 0
  14. 2026LaGuadia: Language-Guided Adaptive Distillation from Pathology Foundation Modelscited by 0
  15. 2026PathReportEval: A Systematic Benchmark for Pathology Report Generationcited by 0
  16. 2026Pathologist Attention-Aligned Report Generation for Prostate Histopathologycited by 0
  17. 2026From Multi-Resolution Cells to Gigapixel Whole Slide Images Foundation Model for Computational Pathologycited by 0
  18. 2026Decomposing Whole Slide Image Report Generation with Graph-Constrained Multiple Instance Learning Workflowscited by 0
  19. 2026SLICEChat: Progressive In-Encoder Token Pruning for Whole-Slide Pathology Language Modelscited by 0