Lesion-Aware Visual-Language Fusion for Automated Image Captioning of Ulcerative Colitis Endoscopic Examinations
Not on the map. Why?
Abstract (arXiv)
We present a lesion-aware image captioning framework for ulcerative colitis (UC). The model integrates ResNet embeddings, Grad-CAM heatmaps, and CBAM-enhanced attention with a T5 decoder. Clinical metadata (MES score 0-3, vascular pattern, bleeding, erythema, friability, ulceration) is injected as natural-language prompts to guide caption generation. The system produces structured, interpretable descriptions aligned with clinical practice and provides MES classification and lesion tags. Compared with baselines, our approach improves caption quality and MES classification accuracy, supporting reliable endoscopic reporting.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 9 of its 19 references that are in the map. First posted on arXiv 3 Sep 2025.
Builds on
9 of 19 references.
References · 9
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2020End-to-End Object Detection with Transformerscited by 7,414
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
- 2014Show and Tell: A Neural Image Caption Generatorcited by 879
- 2020Generating Radiology Reports via Memory-driven Transformercited by 368
- 2019Clinically Accurate Chest X-Ray Report Generationcited by 139
- 2016Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotationcited by 59
- 2020Show, Describe and Conclude: On Exploiting the Structure Information of Chest X-Ray Reportscited by 58
Built on by
Citing papers per year: 2026: 2.
2 papers.