# Deep Triplet Hashing Network for Case-based Medical Image Retrieval

arXiv: https://arxiv.org/abs/2101.12346 · HTML: https://mlsanity.com/paper/2101.12346/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 29 Jan 2021.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Circle Loss: A Unified Perspective of Pair Similarity Optimization](https://mlsanity.com/paper/2002.10857/) (2020)
- [Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images](https://mlsanity.com/paper/1808.08114/) (2018)
- [REFUGE Challenge: A Unified Framework for Evaluating Automated Methods for Glaucoma Assessment from Fundus Photographs](https://mlsanity.com/paper/1910.03667/) (2019)
- [Particular object retrieval with integral max-pooling of CNN activations](https://mlsanity.com/paper/1511.05879/) (2015)
- [A Survey on Learning to Hash](https://mlsanity.com/paper/1606.00185/) (2016)
- [Attention Based Glaucoma Detection: A Large-scale Database and CNN Model](https://mlsanity.com/paper/1903.10831/) (2019)
- [SIFT Meets CNN: A Decade Survey of Instance Retrieval](https://mlsanity.com/paper/1608.01807/) (2016)
- [Feature Learning based Deep Supervised Hashing with Pairwise Labels](https://mlsanity.com/paper/1511.03855/) (2015)
- [An Empirical Study of Spatial Attention Mechanisms in Deep Networks](https://mlsanity.com/paper/1904.05873/) (2019)
- [Simultaneous Feature Learning and Hash Coding with Deep Neural Networks](https://mlsanity.com/paper/1504.03410/) (2015)
- [Deep Supervised Hashing with Triplet Labels](https://mlsanity.com/paper/1612.03900/) (2016)
- [Deep Semantic Ranking Based Hashing for Multi-Label Image Retrieval](https://mlsanity.com/paper/1501.06272/) (2015)
- [Recent Advance in Content-based Image Retrieval: A Literature Survey](https://mlsanity.com/paper/1706.06064/) (2017)
- [AGE Challenge: Angle Closure Glaucoma Evaluation in Anterior Segment Optical Coherence Tomography](https://mlsanity.com/paper/2005.02258/) (2020)
- [Bit-Scalable Deep Hashing with Regularized Similarity Learning for Image Retrieval and Person Re-identification](https://mlsanity.com/paper/1508.04535/) (2015)
- [Fast Training of Triplet-based Deep Binary Embedding Networks](https://mlsanity.com/paper/1603.02844/) (2016)
- [Deeply Activated Salient Region for Instance Search](https://mlsanity.com/paper/2002.00185/) (2020)
- [Instance Search via Instance Level Segmentation and Feature Representation](https://mlsanity.com/paper/1806.03576/) (2018)

## Built on by (most cited first, 8 of 8)

- [Enhancing Diagnostic Accuracy in Rare and Common Fundus Diseases with a Knowledge-Rich Vision-Language Model](https://mlsanity.com/paper/2406.09317/) (2024)
- [Towards All-in-One Medical Image Re-Identification](https://mlsanity.com/paper/2503.08173/) (2025)
- [Pyramid Pixel Context Adaption Network for Medical Image Classification with Supervised Contrastive Learning](https://mlsanity.com/paper/2303.01917/) (2023)
- [Cross-Modality Sub-Image Retrieval using Contrastive Multimodal Image Representations](https://mlsanity.com/paper/2201.03597/) (2022)
- [Efficient Medical Image Retrieval Using DenseNet and FAISS for BIRADS Classification](https://mlsanity.com/paper/2411.01473/) (2024)
- [Revisiting Medical Image Retrieval via Knowledge Consolidation](https://mlsanity.com/paper/2503.09370/) (2025)
- [Clinician-Friendly Foundation Models for Ophthalmic Image Diagnostics without Fine-Tuning or Technical Barriers](https://mlsanity.com/paper/2504.15928/) (2025)
- [CheXtriev: Anatomy-Centered Representation for Case-Based Retrieval of Chest Radiographs](https://mlsanity.com/paper/2608.28137/) (2026)

## Abstract (arXiv)

Deep hashing methods have been shown to be the most efficient approximate nearest neighbor search techniques for large-scale image retrieval. However, existing deep hashing methods have a poor small-sample ranking performance for case-based medical image retrieval. The top-ranked images in the returned query results may be as a different class than the query image. This ranking problem is caused by classification, regions of interest (ROI), and small-sample information loss in the hashing space. To address the ranking problem, we propose an end-to-end framework, called Attention-based Triplet Hashing (ATH) network, to learn low-dimensional hash codes that preserve the classification, ROI, and small-sample information. We embed a spatial-attention module into the network structure of our ATH to focus on ROI information. The spatial-attention module aggregates the spatial information of feature maps by utilizing max-pooling, element-wise maximum, and element-wise mean operations jointly along the channel axis. The triplet cross-entropy loss can help to map the classification information of images and similarity between images into the hash codes. Extensive experiments on two case-based medical datasets demonstrate that our proposed ATH can further improve the retrieval performance compared to the state-of-the-art deep hashing methods and boost the ranking performance for small samples. Compared to the other loss methods, the triplet cross-entropy loss can enhance the classification performance and hash code-discriminability