# Retrieval-Guided Generation for Safer Histopathology Image Captioning

arXiv: https://arxiv.org/abs/2605.00893 · HTML: https://mlsanity.com/paper/2605.00893/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 13 references that are in the map.
- First posted on arXiv 27 Apr 2026.

## Builds on (most cited first, 9 of 9)

- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Phi-4 Technical Report](https://mlsanity.com/paper/2412.08905/) (2024)
- [Virchow2: Scaling Self-Supervised Mixed Magnification Models in Pathology](https://mlsanity.com/paper/2408.00738/) (2024)
- [Multiple Instance Captioning: Learning Representations from Histopathology Textbooks and Articles](https://mlsanity.com/paper/2103.05121/) (2021)
- [Fine-Tuning and Training of DenseNet for Histopathology Image Representation Using TCGA Diagnostic Slides](https://mlsanity.com/paper/2101.07903/) (2021)
- [A General-Purpose Self-Supervised Model for Computational Pathology](https://mlsanity.com/paper/2308.15474/) (2023)
- [Yottixel -- An Image Search Engine for Large Archives of Histopathology Whole Slide Images](https://mlsanity.com/paper/1911.08748/) (2019)
- [Rotation-Agnostic Image Representation Learning for Digital Pathology](https://mlsanity.com/paper/2311.08359/) (2023)

## Abstract (arXiv)

Generative vision-language models can produce fluent medical image captions but remain prone to hallucination, over-specific diagnostic claims, and factual inconsistency-serious issues in pathology. We investigate retrieval-guided generation (RGG) as a safer alternative, where captions are formed by summarizing expert text from visually similar cases rather than generated de novo. On the ARCH histopathology dataset, RGG improves semantic alignment with ground truth, achieving cosine similarity of ≈0.60 versus ≈0.47 from MedGemma, with non-overlapping confidence intervals indicating a robust gain. A pathologist-led qualitative review shows better preservation of morphology-relevant terminology and fewer unsupported diagnoses, while revealing failure modes such as concept mixing and inherited over-specific labeling. Overall, retrieval-guided captioning offers a more transparent and reliable approach with clearer opportunities for auditing than fully generative methods.