# T-DuMpRa: Teacher-guided Dual-path Multi-prototype Retrieval Augmented framework for fine-grained medical image classification

arXiv: https://arxiv.org/abs/2604.17360 · HTML: https://mlsanity.com/paper/2604.17360/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Medical and biomedical RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 43 of its 73 references that are in the map.
- First posted on arXiv 19 Apr 2026.

## Placed in

- Retrieval-augmented generation / [Medical and biomedical RAG](https://mlsanity.com/rag/medical-and-biomedical-rag/)

## Builds on (most cited first, 30 of 43)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [Mixture-of-Experts with Expert Choice Routing](https://mlsanity.com/paper/2202.09368/) (2022)
- [Attention Bottlenecks for Multimodal Fusion](https://mlsanity.com/paper/2107.00135/) (2021)
- [Explainable artificial intelligence (XAI) in deep learning-based medical image analysis](https://mlsanity.com/paper/2107.10912/) (2021)
- [Targeted Supervised Contrastive Learning for Long-Tailed Recognition](https://mlsanity.com/paper/2111.13998/) (2021)
- [Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning](https://mlsanity.com/paper/2105.15134/) (2021)
- [Retrieval Augmented Classification for Long-Tail Visual Recognition](https://mlsanity.com/paper/2202.11233/) (2022)
- [Deep Model Fusion: A Survey](https://mlsanity.com/paper/2309.15698/) (2023)
- [Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels](https://mlsanity.com/paper/2105.04522/) (2021)
- [Explaining the Ambiguity of Object Detection and 6D Pose From Visual Data](https://mlsanity.com/paper/1812.00287/) (2018)
- [PRIOR: Prototype Representation Joint Learning from Medical Images and Reports](https://mlsanity.com/paper/2307.12577/) (2023)
- [Semi-supervised Contrastive Learning for Label-efficient Medical Image Segmentation](https://mlsanity.com/paper/2109.07407/) (2021)
- [Multimodal Generative Learning Utilizing Jensen-Shannon-Divergence](https://mlsanity.com/paper/2006.08242/) (2020)
- [von Mises-Fisher Mixture Model-based Deep learning: Application to Face Verification](https://mlsanity.com/paper/1706.04264/) (2017)
- [Correlation-aware Coarse-to-fine MLPs for Deformable Medical Image Registration](https://mlsanity.com/paper/2406.00123/) (2024)
- [ACPL: Anti-curriculum Pseudo-labelling for Semi-supervised Medical Image Classification](https://mlsanity.com/paper/2111.12918/) (2021)
- [ProtoSeg: Interpretable Semantic Segmentation with Prototypical Parts](https://mlsanity.com/paper/2301.12276/) (2023)
- [Explainable Deep Learning Methods in Medical Image Classification: A Survey](https://mlsanity.com/paper/2205.04766/) (2022)
- [Test-Time Adaptation with SaLIP: A Cascade of SAM and CLIP for Zero shot Medical Image Segmentation](https://mlsanity.com/paper/2404.06362/) (2024)
- [A Recent Survey of Vision Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2312.00634/) (2023)
- [Retrieval-augmented Few-shot Medical Image Segmentation with Foundation Models](https://mlsanity.com/paper/2408.08813/) (2024)

## Built on by (most cited first, 1 of 1)

- [SecondOpinion: Anatomy-Aware Gated Reasoning for Efficient Medical Image Analysis](https://mlsanity.com/paper/2608.01808/) (2026)

## Abstract (arXiv)

Fine-grained medical image classification is challenged by subtle inter-class variations and visually ambiguous cases, where confidence estimates often exhibit uncertainty rather than being overconfident. In such scenarios, purely discriminative classifiers may achieve high overall accuracy yet still fail to distinguish between highly similar categories, leading to miscalibrated predictions. We propose T-DuMpRa, a teacher-guided dual-path multi-prototype retrieval-augmented framework, where discriminative classification and multi-prototype retrieval jointly drive both training and prediction. During training, we jointly optimize cross-entropy and supervised contrastive objectives to learn a cosine-compatible embedding geometry for reliable prototype matching. We further employ an exponential moving average (EMA) teacher to obtain smoother representations and build a multi-prototype memory bank by clustering teacher embeddings in the teacher embedding space. Our framework is plug-and-play: it can be easily integrated into existing classification models by constructing a compact prototype bank, thereby improving performance on visually ambiguous cases. At inference, we combine the classifier's predicted distribution with a similarity-based distribution computed via cosine matching to prototypes, and apply a conservative confidence-gated fusion that activates retrieval only when the classifier's prediction is uncertain and the retrieval evidence is decisive and conflicting, otherwise keeping confident predictions unchanged. On HAM10000 and ISIC2019, our method yields 0.68%-0.21% and 0.44%-2.69% improvements on 5 different backbones. And visualization analysis proves our model can enhance the model's ability to handle visually ambiguous cases.