mlsanityData to 7 Oct 2026

Inference of captions from histopathological patches

arXiv 2202.03432 ↗First posted 7 Feb 2022eess.IV cs.CVcited by 26

Not on the map. Why?

Abstract (arXiv)

Computational histopathology has made significant strides in the past few years, slowly getting closer to clinical adoption. One area of benefit would be the automatic generation of diagnostic reports from H&E-stained whole slide images which would further increase the efficiency of the pathologists' routine diagnostic workflows. In this study, we compiled a dataset (PatchGastricADC22) of histopathological captions of stomach adenocarcinoma endoscopic biopsy specimens, which we extracted from diagnostic reports and paired with patches extracted from the associated whole slide images. The dataset contains a variety of gastric adenocarcinoma subtypes. We trained a baseline attention-based model to predict the captions from features extracted from the patches and obtained promising results. We make the captioned dataset of 262K patches publicly available.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 16 of its 28 references that are in the map. First posted on arXiv 7 Feb 2022.

Builds on

16 of 28 references.

References · 16

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  5. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
  6. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  7. 2019Attention on Attention for Image Captioningcited by 289
  8. 2017On the Automatic Generation of Medical Imaging Reportscited by 215
  9. 2015Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classificationcited by 159
  10. 2014Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)cited by 94
  11. 2016Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotationcited by 59
  12. 2021Multiple Instance Captioning: Learning Representations from Histopathology Textbooks and Articlescited by 58
4 more
  1. 2017MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Networkcited by 56
  2. 2019Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networkscited by 43
  3. 2015Classifying and Segmenting Microscopy Images Using Convolutional Multiple Instance Learningcited by 38
  4. 2017Deep-Learning for Classification of Colorectal Polyps on Whole-Slide Imagescited by 26

Built on by

Cited by year

Citing papers per year: 2023: 10; 2024: 8; 2025: 8.

26 papers.

Sort:

By area:

  1. 2023Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Imagescited by 132
  2. 2024PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathologycited by 80
  3. 2024MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicinecited by 79
  4. 2023Towards a Visual-Language Foundation Model for Computational Pathologycited by 52
  5. 2023Text-guided Foundation Model Adaptation for Pathological Image Classificationcited by 47
  6. 2023CLIP in Medical Imaging: A Surveycited by 40
  7. 2024PathAlign: A vision-language model for whole slide images in histopathologycited by 29
  8. 2023Medical Vision Language Pretraining: A surveycited by 24
  9. 2023A Foundational Multimodal Vision Language AI Assistant for Human Pathologycited by 19
  10. 2024PathM3: A Multimodal Multi-Task Multiple Instance Learning Framework for Whole Slide Image Classification and Captioningcited by 13
  11. 2023Path to Medical AGI: Unify Domain-specific Medical LLMs with the Lowest Costcited by 12
  12. 2025UniBiomed: A Universal Foundation Model for Grounded Biomedical Image Interpretationcited by 10
14 more
  1. 2023Automatic Report Generation for Histopathology images using pre-trained Vision Transformerscited by 9
  2. 2024A New Era in Computational Pathology: A Survey on Foundation and Vision-Language Modelscited by 9
  3. 2023What a Whole Slide Image Can Tell? Subtype-guided Masked Transformer for Pathological Image Captioningcited by 8
  4. 2025How Good is my Histopathology Vision-Language Foundation Model? A Holistic Benchmarkcited by 7
  5. 2025Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challengescited by 5
  6. 2024Enhancing Multimodal Large Language Models with Multi-instance Visual Prompt Generator for Visual Representation Enrichmentcited by 4
  7. 2025MGPATH: Vision-Language Model with Multi-Granular Prompt Learning for Few-Shot WSI Classificationcited by 3
  8. 2024SeLoRA: Self-Expanding Low-Rank Adaptation of Latent Diffusion Model for Medical Image Synthesiscited by 2
  9. 2024MECFormer: Multi-task Whole Slide Image Classification with Expert Consultation Networkcited by 2
  10. 2025Slide-Level Prompt Learning with Vision Language Models for Few-Shot Multiple Instance Learning in Histopathologycited by 1
  11. 2025HeteroRAG: A Heterogeneous Retrieval-Augmented Generation Framework for Medical Vision Language Taskscited by 1
  12. 2023Automatic Report Generation for Histopathology images using pre-trained Vision Transformers and BERTcited by 0
  13. 2025Any-to-Any Learning in Computational Pathology via Triplet Multimodal Pretrainingcited by 0
  14. 2025GNN-ViTCap: GNN-Enhanced Multiple Instance Learning with Vision Transformers for Whole Slide Image Classification and Captioningcited by 0