mlsanityData to 14 Oct 2026

Histopathology Image Report Generation by Vision Language Model with Multimodal In-Context Learning

arXiv 2506.17645 ↗First posted 21 Jun 2025cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Automating medical report generation from histopathology images is a critical challenge requiring effective visual representations and domain-specific knowledge. Inspired by the common practices of human experts, we propose an in-context learning framework called PathGenIC that integrates context derived from the training set with a multimodal in-context learning (ICL) mechanism. Our method dynamically retrieves semantically similar whole slide image (WSI)-report pairs and incorporates adaptive feedback to enhance contextual relevance and generation quality. Evaluated on the HistGen benchmark, the framework achieves state-of-the-art results, with significant improvements across BLEU, METEOR, and ROUGE-L metrics, and demonstrates robustness across diverse report lengths and disease categories. By maximizing training data utility and bridging vision and language with ICL, our work offers a solution for AI-driven histopathology reporting, setting a strong foundation for future advancements in multimodal clinical applications.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 11 of its 14 references that are in the map. First posted on arXiv 21 Jun 2025.

Builds on

11 of 14 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 10

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  6. 2020Generating Radiology Reports via Memory-driven Transformercited by 368
  7. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  8. 2022Cross-modal Memory Networks for Radiology Report Generationcited by 185
  9. 2020Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generationcited by 141
  10. 2023Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videoscited by 86

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Taming Visual Neglect: A Variational Information Bottleneck Framework for Adaptive Attention in Multimodal In-Context Learningcited by 0
  2. 2026Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024PathAlign: A vision-language model for whole slide images in histopathology
  2. 2025PolyPath: Adapting a Large Multimodal Model for Multi-slide Pathology Report Generation
  3. 2026Semantic Context-aware mOdality fUsion Transformer (SCOUT): A Context-Aware Multimodal Transformer for Concept-Grounded Pathology Report Generation
  4. 2025MPath: Multimodal Pathology Report Generation from Whole Slide Images
  5. 2025On the Importance of Text Preprocessing for Multimodal Representation Learning and Pathology Report Generation
  6. 2024Clinical-grade Multi-Organ Pathology Report Generation for Multi-scale Whole Slide Images via a Semantically Guided Medical Text Foundation Model
  7. 2026Footprint-Guided Exemplar-Free Continual Histopathology Report Generation
  8. 2025ChatEXAONEPath: An Expert-level Multimodal Large Language Model for Histopathology Using Whole Slide Images