# Report on the AAPM Grand Challenge on deep generative modeling for learning medical image statistics arXiv: https://arxiv.org/abs/2405.01822 ยท HTML: https://mlsanity.com/paper/2405.01822/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Builds on 28 of its 76 references that are in the map. - First posted on arXiv 3 May 2024. ## Builds on (most cited first, 28 of 28) - [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014) - [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021) - [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019) - [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020) - [Solving Inverse Problems in Medical Imaging with Score-Based Generative Models](https://mlsanity.com/paper/2111.08005/) (2021) - [Tackling the Generative Learning Trilemma with Denoising Diffusion GANs](https://mlsanity.com/paper/2112.07804/) (2021) - [StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets](https://mlsanity.com/paper/2202.00273/) (2022) - [Reliable Fidelity and Diversity Metrics for Generative Models](https://mlsanity.com/paper/2002.09797/) (2020) - [A note on the evaluation of generative models](https://mlsanity.com/paper/1511.01844/) (2015) - [Generative Adversarial Network in Medical Imaging: A Review](https://mlsanity.com/paper/1809.07294/) (2018) - [Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models](https://mlsanity.com/paper/2306.04675/) (2023) - [Pros and Cons of GAN Evaluation Measures: New Developments](https://mlsanity.com/paper/2103.09396/) (2021) - [Distribution Matching Losses Can Hallucinate Features in Medical Image Translation](https://mlsanity.com/paper/1805.08841/) (2018) - [Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task](https://mlsanity.com/paper/2310.09336/) (2023) - [GANs for Medical Image Analysis](https://mlsanity.com/paper/1809.06222/) (2018) - [Diffusion Probabilistic Models beat GANs on Medical Images](https://mlsanity.com/paper/2212.07501/) (2022) - [On hallucinations in tomographic image reconstruction](https://mlsanity.com/paper/2012.00646/) (2020) - [Overcoming Barriers to Data Sharing with Medical Image Generation: A Comprehensive Evaluation](https://mlsanity.com/paper/2012.03769/) (2020) - [Bayesian Image Reconstruction using Deep Generative Models](https://mlsanity.com/paper/2012.04567/) (2020) - [A Review on Deep Learning in Medical Image Reconstruction](https://mlsanity.com/paper/1906.10643/) (2019) - [Approximating the Ideal Observer and Hotelling Observer for binary signal detection tasks by use of supervised learning methods](https://mlsanity.com/paper/1905.06330/) (2019) - [Learning stochastic object models from medical imaging measurements by use of advanced ambient generative adversarial networks](https://mlsanity.com/paper/2106.14324/) (2021) - [Assessing the ability of generative adversarial networks to learn canonical medical image statistics](https://mlsanity.com/paper/2204.12007/) (2022) - [Evaluating the Performance of StyleGAN2-ADA on Medical Images](https://mlsanity.com/paper/2210.03786/) (2022) - [Assessing the capacity of a denoising diffusion probabilistic model to reproduce spatial context](https://mlsanity.com/paper/2309.10817/) (2023) - [Breast density classification with deep convolutional neural networks](https://mlsanity.com/paper/1711.03674/) (2017) - [Metrics to Quantify Global Consistency in Synthetic Medical Images](https://mlsanity.com/paper/2308.00402/) (2023) - [A Method for Evaluating Deep Generative Models of Images via Assessing the Reproduction of High-order Spatial Context](https://mlsanity.com/paper/2111.12577/) (2021) ## Built on by (most cited first, 4 of 4) - [Rapid Review of Generative AI in Smart Medical Applications](https://mlsanity.com/paper/2406.06627/) (2024) - [Quantifying the uncertainty of model-based synthetic image quality metrics](https://mlsanity.com/paper/2504.03623/) (2025) - [Evaluation of Machine-generated Biomedical Images via A Tally-based Similarity Measure](https://mlsanity.com/paper/2503.22658/) (2025) - [Knowledge-based anomaly detection for identifying network-induced shape artifacts](https://mlsanity.com/paper/2511.04729/) (2025) ## Abstract (arXiv) The findings of the 2023 AAPM Grand Challenge on Deep Generative Modeling for Learning Medical Image Statistics are reported in this Special Report. The goal of this challenge was to promote the development of deep generative models (DGMs) for medical imaging and to emphasize the need for their domain-relevant assessment via the analysis of relevant image statistics. As part of this Grand Challenge, a training dataset was developed based on 3D anthropomorphic breast phantoms from the VICTRE virtual imaging toolbox. A two-stage evaluation procedure consisting of a preliminary check for memorization and image quality (based on the Frechet Inception distance (FID)), and a second stage evaluating the reproducibility of image statistics corresponding to domain-relevant radiomic features was developed. A summary measure was employed to rank the submissions. Additional analyses of submissions was performed to assess DGM performance specific to individual feature families, and to identify various artifacts. 58 submissions from 12 unique users were received for this Challenge. The top-ranked submission employed a conditional latent diffusion model, whereas the joint runners-up employed a generative adversarial network, followed by another network for image superresolution. We observed that the overall ranking of the top 9 submissions according to our evaluation method (i) did not match the FID-based ranking, and (ii) differed with respect to individual feature families. Another important finding from our additional analyses was that different DGMs demonstrated similar kinds of artifacts. This Grand Challenge highlighted the need for domain-specific evaluation to further DGM design as well as deployment. It also demonstrated that the specification of a DGM may differ depending on its intended use.