# Integrating AI into Radiology workflow: Levels of research, production, and feedback maturity

arXiv: https://arxiv.org/abs/1910.06424 · HTML: https://mlsanity.com/paper/1910.06424/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2019.

## Built on by (most cited first, 7 of 7)

- [MedVAE: Efficient Automated Interpretation of Medical Images with Large-Scale Generalizable Autoencoders](https://mlsanity.com/paper/2502.14753/) (2025)
- [CheXstray: Real-time Multi-Modal Data Concordance for Drift Detection in Medical Imaging AI](https://mlsanity.com/paper/2202.02833/) (2022)
- [An Orchestration Platform that Puts Radiologists in the Driver's Seat of AI Innovation: A Methodological Approach](https://mlsanity.com/paper/2107.04409/) (2021)
- [Advancing Brain Metastases Detection in T1-Weighted Contrast-Enhanced 3D MRI using Noisy Student-based Training](https://mlsanity.com/paper/2111.05959/) (2021)
- [Examining Deployment and Refinement of the VIOLA-AI Intracranial Hemorrhage Model Using an Interactive NeoMedSys Platform](https://mlsanity.com/paper/2505.09380/) (2025)
- [IODeep: an IOD for the introduction of deep learning in the DICOM standard](https://mlsanity.com/paper/2311.16163/) (2023)
- [LLaMA-XR: A Novel Framework for Radiology Report Generation using LLaMA and QLoRA Fine Tuning](https://mlsanity.com/paper/2506.03178/) (2025)

## Abstract (arXiv)

This report represents a roadmap for integrating Artificial Intelligence (AI)-based image analysis algorithms into existing Radiology workflows such that: (1) radiologists can significantly benefit from enhanced automation in various imaging tasks due to AI; and (2) radiologists' feedback is utilized to further improve the AI application. This is achieved by establishing three maturity levels where: (1) research enables the visualization of AI-based results/annotations by radiologists without generating new patient records; (2) production allows the AI-based system to generate results stored in an institution's Picture Archiving and Communication System; and (3) feedback equips radiologists with tools for editing the AI inference results for periodic retraining of the deployed AI systems, thereby allowing the continuous organic improvement of AI-based radiology-workflow solutions. A case study (i.e., detection of brain metastases with T1-weighted contrast-enhanced 3D MRI) illustrates the deployment details of a particular AI-based application according to the aforementioned maturity levels. It is shown that the given AI application significantly improves with the feedback coming from radiologists; the number of incorrectly detected brain metastases (false positives) reduces from 14.2 to 9.12 per patient with the number of subsequently annotated datasets increasing from 93 to 217 as a result of radiologist adjudication.