# Pathology Report Generation and Multimodal Representation Learning for Cutaneous Melanocytic Lesions

arXiv: https://arxiv.org/abs/2502.19293 · HTML: https://mlsanity.com/paper/2502.19293/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 27 references that are in the map.
- First posted on arXiv 26 Feb 2025.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://mlsanity.com/paper/2205.01917/) (2022)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [Multimodal Whole Slide Foundation Model for Pathology](https://mlsanity.com/paper/2411.19666/) (2024)
- [PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathology](https://mlsanity.com/paper/2405.10254/) (2024)
- [PathAlign: A vision-language model for whole slide images in histopathology](https://mlsanity.com/paper/2406.19578/) (2024)
- [Tissue Cross-Section and Pen Marking Segmentation in Whole Slide Images](https://mlsanity.com/paper/2401.13511/) (2024)
- [Artificial Intelligence-Based Triaging of Cutaneous Melanocytic Lesions](https://mlsanity.com/paper/2410.10509/) (2024)

## Built on by (most cited first, 6 of 6)

- [Multi-Modal Foundation Models for Computational Pathology: A Survey](https://mlsanity.com/paper/2503.09091/) (2025)
- [Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challenges](https://mlsanity.com/paper/2505.10993/) (2025)
- [RANGER: Sparsely-Gated Mixture-of-Experts with Adaptive Retrieval Re-ranking for Pathology Report Generation](https://mlsanity.com/paper/2603.04348/) (2026)
- [Democratising Pathology Co-Pilots: An Open Pipeline and Dataset for Whole-Slide Vision-Language Modelling](https://mlsanity.com/paper/2512.17326/) (2025)
- [Semantic Context-aware mOdality fUsion Transformer (SCOUT): A Context-Aware Multimodal Transformer for Concept-Grounded Pathology Report Generation](https://mlsanity.com/paper/2605.01144/) (2026)
- [PathReportEval: A Systematic Benchmark for Pathology Report Generation](https://mlsanity.com/paper/2607.18448/) (2026)

## Abstract (arXiv)

Millions of melanocytic skin lesions are examined by pathologists each year, the majority of which concern common nevi (i.e., ordinary moles). While most of these lesions can be diagnosed in seconds, writing the corresponding pathology report is much more time-consuming. Automating part of the report writing could, therefore, alleviate the increasing workload of pathologists. In this work, we develop a vision-language model specifically for the pathology domain of cutaneous melanocytic lesions. The model follows the Contrastive Captioner framework and was trained and evaluated using a melanocytic lesion dataset of 42,512 H&E-stained whole slide images and 19,645 corresponding pathology reports. Our results show that the quality scores of model-generated reports were on par with pathologist-written reports for common nevi, assessed by an expert pathologist in a reader study. While report generation revealed to be more difficult for rare melanocytic lesion subtypes, the cross-modal retrieval performance for these cases was considerably better.