# Medical Image Retrieval using Deep Convolutional Neural Network

arXiv: https://arxiv.org/abs/1703.08472 · HTML: https://mlsanity.com/paper/1703.08472/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Mar 2017.

## Built on by (most cited first, 30 of 33)

- [Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-modal Pretraining](https://mlsanity.com/paper/2107.14572/) (2021)
- [Compatibility-aware Heterogeneous Visual Search](https://mlsanity.com/paper/2105.06047/) (2021)
- [3D-MIR: A Benchmark and Empirical Study on 3D Medical Image Retrieval in Radiology](https://mlsanity.com/paper/2311.13752/) (2023)
- [On Image Search in Histopathology](https://mlsanity.com/paper/2401.08699/) (2024)
- [Learning A 3D-CNN and Transformer Prior for Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/2111.13923/) (2021)
- [X-TRA: Improving Chest X-ray Tasks with Cross-Modal Retrieval Augmentation](https://mlsanity.com/paper/2302.11352/) (2023)
- [Medical Image Retrieval Using Pretrained Embeddings](https://mlsanity.com/paper/2311.13547/) (2023)
- [Region-based Contrastive Pretraining for Medical Image Retrieval with Anatomic Query](https://mlsanity.com/paper/2305.05598/) (2023)
- [Content-Based Image Retrieval for Multi-Class Volumetric Radiology Images: A Benchmark Study](https://mlsanity.com/paper/2405.09334/) (2024)
- [RadIR: A Scalable Framework for Multi-Grained Medical Image Retrieval via Radiology Report Mining](https://mlsanity.com/paper/2503.04653/) (2025)
- [Cross-Modality Sub-Image Retrieval using Contrastive Multimodal Image Representations](https://mlsanity.com/paper/2201.03597/) (2022)
- [Medical Image Retrieval via Nearest Neighbor Search on Pre-trained Image Features](https://mlsanity.com/paper/2210.02401/) (2022)
- [Efficient Medical Image Retrieval Using DenseNet and FAISS for BIRADS Classification](https://mlsanity.com/paper/2411.01473/) (2024)
- [Searching for Pneumothorax in X-Ray Images Using Autoencoded Deep Features](https://mlsanity.com/paper/2102.06096/) (2021)
- [Entity-Graph Enhanced Cross-Modal Pretraining for Instance-level Product Retrieval](https://mlsanity.com/paper/2206.08842/) (2022)
- [Siamese Content-based Search Engine for a More Transparent Skin and Breast Cancer Diagnosis through Histological Imaging](https://mlsanity.com/paper/2401.08272/) (2024)
- [M3Ret: Unleashing Zero-shot Multimodal Medical Image Retrieval via Self-Supervision](https://mlsanity.com/paper/2509.01360/) (2025)
- [Deep Learning Based Decision Support for Medicine -- A Case Study on Skin Cancer Diagnosis](https://mlsanity.com/paper/2103.05112/) (2021)
- [A Similarity Measure of Histopathology Images by Deep Embeddings](https://mlsanity.com/paper/2107.13703/) (2021)
- [Efficient and Discriminative Image Feature Extraction for Universal Image Retrieval](https://mlsanity.com/paper/2409.13513/) (2024)
- [RadiomicsRetrieval: A Customizable Framework for Medical Image Retrieval Using Radiomics Features](https://mlsanity.com/paper/2507.08546/) (2025)
- [Content-based 3D Image Retrieval and a ColBERT-inspired Re-ranking for Tumor Flagging and Staging](https://mlsanity.com/paper/2507.17412/) (2025)
- [Identifying partial mouse brain microscopy images from Allen reference atlas using a contrastively learned semantic space](https://mlsanity.com/paper/2109.06662/) (2021)
- [Exploring Content Based Image Retrieval for Highly Imbalanced Melanoma Data using Style Transfer, Semantic Image Segmentation and Ensemble Learning](https://mlsanity.com/paper/2110.06331/) (2021)
- [Indicative Image Retrieval: Turning Blackbox Learning into Grey](https://mlsanity.com/paper/2201.11898/) (2022)
- [OSCARS: An Outlier-Sensitive Content-Based Radiography Retrieval System](https://mlsanity.com/paper/2204.03074/) (2022)
- [Anatomy-Aware Conditional Image-Text Retrieval](https://mlsanity.com/paper/2503.07456/) (2025)
- [Advanced Multi-Architecture Deep Learning Framework for BIRADS-Based Mammographic Image Retrieval: Comprehensive Performance Analysis with Super-Ensemble Optimization](https://mlsanity.com/paper/2508.04790/) (2025)
- [J-RAS: Mutual Adaptation for Medical Image Segmentation via Contrastive Retrieval-Augmented Joint Optimization](https://mlsanity.com/paper/2510.09953/) (2025)
- [mViSE: A Visual Search Engine for Analyzing Multiplex IHC Brain Tissue Images](https://mlsanity.com/paper/2512.11745/) (2025)

## Abstract (arXiv)

With a widespread use of digital imaging data in hospitals, the size of medical image repositories is increasing rapidly. This causes difficulty in managing and querying these large databases leading to the need of content based medical image retrieval (CBMIR) systems. A major challenge in CBMIR systems is the semantic gap that exists between the low level visual information captured by imaging devices and high level semantic information perceived by human. The efficacy of such systems is more crucial in terms of feature representations that can characterize the high-level information completely. In this paper, we propose a framework of deep learning for CBMIR system by using deep Convolutional Neural Network (CNN) that is trained for classification of medical images. An intermodal dataset that contains twenty four classes and five modalities is used to train the network. The learned features and the classification results are used to retrieve medical images. For retrieval, best results are achieved when class based predictions are used. An average classification accuracy of 99.77% and a mean average precision of 0.69 is achieved for retrieval task. The proposed method is best suited to retrieve multimodal medical images for different body organs.