# Objective task-based evaluation of artificial intelligence-based medical imaging methods: Framework, strategies and role of the physician

arXiv: https://arxiv.org/abs/2107.04540 · HTML: https://mlsanity.com/paper/2107.04540/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jul 2021.

## Built on by (most cited first, 16 of 16)

- [Metrics that matter: Evaluating image quality metrics for medical image generation](https://mlsanity.com/paper/2505.07175/) (2025)
- [Assessing the ability of generative adversarial networks to learn canonical medical image statistics](https://mlsanity.com/paper/2204.12007/) (2022)
- [A task-specific deep-learning-based denoising approach for myocardial perfusion SPECT](https://mlsanity.com/paper/2303.00212/) (2023)
- [Investigating the limited performance of a deep-learning-based SPECT denoising approach: An observer-study-based characterization](https://mlsanity.com/paper/2203.01918/) (2022)
- [Need for Objective Task-based Evaluation of Deep Learning-Based Denoising Methods: A Study in the Context of Myocardial Perfusion SPECT](https://mlsanity.com/paper/2303.02110/) (2023)
- [Prior image-based medical image reconstruction using a style-based generative adversarial network](https://mlsanity.com/paper/2202.08936/) (2022)
- [Evaluating Procedures for Establishing Generative Adversarial Network-based Stochastic Image Models in Medical Imaging](https://mlsanity.com/paper/2204.03547/) (2022)
- [DEMIST: A deep-learning-based task-specific denoising approach for myocardial perfusion SPECT](https://mlsanity.com/paper/2306.04249/) (2023)
- [Nuclear Medicine AI in Action: The Bethesda Report (AI Summit 2024)](https://mlsanity.com/paper/2406.01044/) (2024)
- [What is Implementation Science; and Why It Matters for Bridging the Artificial Intelligence Innovation-to-Application Gap in Medical Imaging](https://mlsanity.com/paper/2510.13006/) (2025)
- [Issues and Challenges in Applications of Artificial Intelligence to Nuclear Medicine -- The Bethesda Report (AI Summit 2022)](https://mlsanity.com/paper/2211.03783/) (2022)
- [A quality assurance framework for real-time monitoring of deep learning segmentation models in radiotherapy](https://mlsanity.com/paper/2305.11715/) (2023)
- [Objective Task-based Evaluation of Quantitative Medical Imaging Methods: Emerging Frameworks and Future Directions](https://mlsanity.com/paper/2507.04591/) (2025)
- [Benchmarking Deep Learning-Based Reconstruction Methods for Photoacoustic Computed Tomography with Clinically Relevant Synthetic Datasets](https://mlsanity.com/paper/2601.17165/) (2026)
- [Rethinking Artificial Intelligence in Medical Imaging: Assumptions, Reality, and Reframing](https://mlsanity.com/paper/2607.27428/) (2026)
- [Automated carotid artery atherosclerotic inflammation segmentation in PET using MSR-Net](https://mlsanity.com/paper/2610.15245/) (2026)

## Abstract (arXiv)

Artificial intelligence (AI)-based methods are showing promise in multiple medical-imaging applications. Thus, there is substantial interest in clinical translation of these methods, requiring in turn, that they be evaluated rigorously. In this paper, our goal is to lay out a framework for objective task-based evaluation of AI methods. We will also provide a list of tools available in the literature to conduct this evaluation. Further, we outline the important role of physicians in conducting these evaluation studies. The examples in this paper will be proposed in the context of PET with a focus on neural-network-based methods. However, the framework is also applicable to evaluate other medical-imaging modalities and other types of AI methods.