# Deep reinforcement learning with automated label extraction from clinical reports accurately classifies 3D MRI brain volumes

arXiv: https://arxiv.org/abs/2106.09812 · HTML: https://mlsanity.com/paper/2106.09812/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 36 references that are in the map.
- First posted on arXiv 17 Jun 2021.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Deep learning in radiology: an overview of the concepts and a survey of the state of the art](https://mlsanity.com/paper/1802.08717/) (2018)
- [Deep reinforcement learning in medical imaging: A literature review](https://mlsanity.com/paper/2103.05115/) (2021)
- [Deep reinforcement learning to detect brain lesions on MRI: a proof-of-concept application of reinforcement learning to medical images](https://mlsanity.com/paper/2008.02708/) (2020)
- [Partial Policy-based Reinforcement Learning for Anatomical Landmark Localization in 3D Medical Images](https://mlsanity.com/paper/1807.02908/) (2018)
- [Reinforcement learning using Deep Q Networks and Q learning accurately localizes brain tumors on MRI with very small training sets](https://mlsanity.com/paper/2010.10763/) (2020)
- [Unsupervised deep clustering and reinforcement learning can accurately segment MRI brain tumors with very small training sets](https://mlsanity.com/paper/2012.13321/) (2020)
- [Deep reinforcement learning-based image classification achieves perfect testing set accuracy for MRI brain tumors with a training set of only 30 images](https://mlsanity.com/paper/2102.02895/) (2021)

## Built on by (most cited first, 12 of 12)

- [Reinforcement Learning in Medical Image Analysis: Concepts, Applications, Challenges, and Future Directions](https://mlsanity.com/paper/2206.14302/) (2022)
- [Enhancing chest X-ray datasets with privacy-preserving large language models and multi-type annotations: a data-driven approach for improved classification](https://mlsanity.com/paper/2403.04024/) (2024)
- [Selective experience replay compression using coresets for lifelong deep reinforcement learning in medical imaging](https://mlsanity.com/paper/2302.11510/) (2023)
- [Deep Neuroevolution Squeezes More out of Small Neural Networks and Small Training Sets: Sample Application to MRI Brain Sequence Classification](https://mlsanity.com/paper/2112.12990/) (2021)
- [Deep reinforcement learning for fMRI prediction of Autism Spectrum Disorder](https://mlsanity.com/paper/2206.11224/) (2022)
- [Deep neuroevolution for limited, heterogeneous data: proof-of-concept application to Neuroblastoma brain metastasis using a small virtual pooled image collection](https://mlsanity.com/paper/2211.14499/) (2022)
- [Application Of Vision-Language Models For Assessing Osteoarthritis Disease Severity](https://mlsanity.com/paper/2401.06331/) (2024)
- [Large Language Model-Based Uncertainty-Adjusted Label Extraction for Artificial Intelligence Model Development in Upper Extremity Radiography](https://mlsanity.com/paper/2510.05664/) (2025)
- [Understanding the effect of varying amounts of replay per step](https://mlsanity.com/paper/2302.10311/) (2023)
- [Multi-environment lifelong deep reinforcement learning for medical imaging](https://mlsanity.com/paper/2306.00188/) (2023)
- [A framework for dynamically training and adapting deep reinforcement learning models to different, low-compute, and continuously changing radiology deployment environments](https://mlsanity.com/paper/2306.05310/) (2023)
- [Kinematics Modeling of Peroxy Free Radicals: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2404.10010/) (2024)

## Abstract (arXiv)

Purpose: Image classification is perhaps the most fundamental task in imaging AI. However, labeling images is time-consuming and tedious. We have recently demonstrated that reinforcement learning (RL) can classify 2D slices of MRI brain images with high accuracy. Here we make two important steps toward speeding image classification: Firstly, we automatically extract class labels from the clinical reports. Secondly, we extend our prior 2D classification work to fully 3D image volumes from our institution. Hence, we proceed as follows: in Part 1, we extract labels from reports automatically using the SBERT natural language processing approach. Then, in Part 2, we use these labels with RL to train a classification Deep-Q Network (DQN) for 3D image volumes. Methods: For Part 1, we trained SBERT with 90 radiology report impressions. We then used the trained SBERT to predict class labels for use in Part 2. In Part 2, we applied multi-step image classification to allow for combined Deep-Q learning using 3D convolutions and TD(0) Q learning. We trained on a set of 90 images. We tested on a separate set of 61 images, again using the classes predicted from patient reports by the trained SBERT in Part 1. For comparison, we also trained and tested a supervised deep learning classification network on the same set of training and testing images using the same labels. Results: Part 1: Upon training with the corpus of radiology reports, the SBERT model had 100% accuracy for both normal and metastasis-containing scans. Part 2: Then, using these labels, whereas the supervised approach quickly overfit the training data and as expected performed poorly on the testing set (66% accuracy, just over random guessing), the reinforcement learning approach achieved an accuracy of 92%. The results were found to be statistically significant, with a p-value of 3.1 x 10^-5.