# Deep Metric Learning-based Image Retrieval System for Chest Radiograph and its Clinical Applications in COVID-19

arXiv: https://arxiv.org/abs/2012.03663 · HTML: https://mlsanity.com/paper/2012.03663/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Nov 2020.

## Built on by (most cited first, 12 of 12)

- [Superficial White Matter Analysis: An Efficient Point-cloud-based Deep Learning Framework with Supervised Contrastive Learning for Consistent Tractography Parcellation across Populations and dMRI Acquisitions](https://mlsanity.com/paper/2207.08975/) (2022)
- [Augmenting Medical Imaging: A Comprehensive Catalogue of 65 Techniques for Enhanced Data Analysis](https://mlsanity.com/paper/2303.01178/) (2023)
- [Sketch-based Medical Image Retrieval](https://mlsanity.com/paper/2303.03633/) (2023)
- [Region-based Contrastive Pretraining for Medical Image Retrieval with Anatomic Query](https://mlsanity.com/paper/2305.05598/) (2023)
- [Medical Image Retrieval via Nearest Neighbor Search on Pre-trained Image Features](https://mlsanity.com/paper/2210.02401/) (2022)
- [OSCARS: An Outlier-Sensitive Content-Based Radiography Retrieval System](https://mlsanity.com/paper/2204.03074/) (2022)
- [Attention-based Dynamic Subspace Learners for Medical Image Analysis](https://mlsanity.com/paper/2206.09068/) (2022)
- [Multimorbidity Content-Based Medical Image Retrieval Using Proxies](https://mlsanity.com/paper/2211.12185/) (2022)
- [Spatially Optimized Compact Deep Metric Learning Model for Similarity Search](https://mlsanity.com/paper/2404.06593/) (2024)
- [From Lab to Pocket: A Novel Continual Learning-based Mobile Application for Screening COVID-19](https://mlsanity.com/paper/2410.12589/) (2024)
- [Feature Subset Weighting for Distance-based Supervised Learning through Choquet Integration](https://mlsanity.com/paper/2504.00624/) (2025)
- [WristMIR: Coarse-to-Fine Region-Aware Retrieval of Pediatric Wrist Radiographs with Radiology Report-Driven Learning](https://mlsanity.com/paper/2602.07872/) (2026)

## Abstract (arXiv)

In recent years, deep learning-based image analysis methods have been widely applied in computer-aided detection, diagnosis and prognosis, and has shown its value during the public health crisis of the novel coronavirus disease 2019 (COVID-19) pandemic. Chest radiograph (CXR) has been playing a crucial role in COVID-19 patient triaging, diagnosing and monitoring, particularly in the United States. Considering the mixed and unspecific signals in CXR, an image retrieval model of CXR that provides both similar images and associated clinical information can be more clinically meaningful than a direct image diagnostic model. In this work we develop a novel CXR image retrieval model based on deep metric learning. Unlike traditional diagnostic models which aims at learning the direct mapping from images to labels, the proposed model aims at learning the optimized embedding space of images, where images with the same labels and similar contents are pulled together. It utilizes multi-similarity loss with hard-mining sampling strategy and attention mechanism to learn the optimized embedding space, and provides similar images to the query image. The model is trained and validated on an international multi-site COVID-19 dataset collected from 3 different sources. Experimental results of COVID-19 image retrieval and diagnosis tasks show that the proposed model can serve as a robust solution for CXR analysis and patient management for COVID-19. The model is also tested on its transferability on a different clinical decision support task, where the pre-trained model is applied to extract image features from a new dataset without any further training. These results demonstrate our deep metric learning based image retrieval model is highly efficient in the CXR retrieval, diagnosis and prognosis, and thus has great clinical value for the treatment and management of COVID-19 patients.