mlsanityData to 14 Oct 2026

Building RadiologyNET: Unsupervised annotation of a large-scale multimodal medical database

arXiv 2308.08517 ↗First posted 27 Jul 2023cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Background and objective: The usage of machine learning in medical diagnosis and treatment has witnessed significant growth in recent years through the development of computer-aided diagnosis systems that are often relying on annotated medical radiology images. However, the availability of large annotated image datasets remains a major obstacle since the process of annotation is time-consuming and costly. This paper explores how to automatically annotate a database of medical radiology images with regard to their semantic similarity. Material and methods: An automated, unsupervised approach is used to construct a large annotated dataset of medical radiology images originating from Clinical Hospital Centre Rijeka, Croatia, utilising multimodal sources, including images, DICOM metadata, and narrative diagnoses. Several appropriate feature extractors are tested for each of the data sources, and their utility is evaluated using k-means and k-medoids clustering on a representative data subset. Results: The optimal feature extractors are then integrated into a multimodal representation, which is then clustered to create an automated pipeline for labelling a precursor dataset of 1,337,926 medical images into 50 clusters of visually similar images. The quality of the clusters is assessed by examining their homogeneity and mutual information, taking into account the anatomical region and modality representation. Conclusion: The results suggest that fusing the embeddings of all three data sources together works best for the task of unsupervised clustering of large-scale medical data, resulting in the most concise clusters. Hence, this work is the first step towards building a much larger and more fine-grained annotated dataset of medical radiology images.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 52 references that are in the map. First posted on arXiv 27 Jul 2023.

Builds on

19 of 52 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  7. 2018Attention U-Net: Learning Where to Look for the Pancreascited by 1,435
  8. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  9. 2014Distributed Representations of Sentences and Documentscited by 678
  10. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 664
  11. 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 368
  12. 2019Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentationcited by 260
7 more
  1. 2019Multilingual Universal Sentence Encoder for Semantic Retrievalcited by 193
  2. 2018Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentationcited by 180
  3. 2019Text Classification Algorithms: A Surveycited by 163
  4. 2020A Survey on Text Classification: From Shallow to Deep Learningcited by 80
  5. 2021Supervised Transfer Learning at Scale for Medical Imagingcited by 38
  6. 2020Sentiment Analysis Based on Deep Learning: A Comparative Studycited by 37
  7. 2017Random Forest Missing Data Algorithmscited by 23

Built on by

Not cited yet.