# Scorecards for Synthetic Medical Data Evaluation and Reporting

arXiv: https://arxiv.org/abs/2406.11143 · HTML: https://mlsanity.com/paper/2406.11143/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 29 references that are in the map.
- First posted on arXiv 17 Jun 2024.

## Builds on (most cited first, 13 of 13)

- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Reliable Fidelity and Diversity Metrics for Generative Models](https://mlsanity.com/paper/2002.09797/) (2020)
- [The Vendi Score: A Diversity Evaluation Metric for Machine Learning](https://mlsanity.com/paper/2210.02410/) (2022)
- [Pros and Cons of GAN Evaluation Measures](https://mlsanity.com/paper/1802.03446/) (2018)
- [Pros and Cons of GAN Evaluation Measures: New Developments](https://mlsanity.com/paper/2103.09396/) (2021)
- [Detecting and Evaluating Medical Hallucinations in Large Vision Language Models](https://mlsanity.com/paper/2406.10185/) (2024)
- [Overcoming Barriers to Data Sharing with Medical Image Generation: A Comprehensive Evaluation](https://mlsanity.com/paper/2012.03769/) (2020)
- [Healthsheet: Development of a Transparency Artifact for Health Datasets](https://mlsanity.com/paper/2202.13028/) (2022)
- [Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Design](https://mlsanity.com/paper/2302.02913/) (2023)
- [Constraining Generative Models for Engineering Design with Negative Data](https://mlsanity.com/paper/2306.15166/) (2023)
- [Assessing the capacity of a denoising diffusion probabilistic model to reproduce spatial context](https://mlsanity.com/paper/2309.10817/) (2023)
- [Synthetic Data in Radiological Imaging: Current State and Future Outlook](https://mlsanity.com/paper/2407.01561/) (2024)

## Built on by (most cited first, 4 of 4)

- [CheXGenBench: A Unified Benchmark For Fidelity, Privacy and Utility of Synthetic Chest Radiographs](https://mlsanity.com/paper/2505.10496/) (2025)
- [Generating Reliable Synthetic Clinical Trial Data: The Role of Hyperparameter Optimization and Domain Constraints](https://mlsanity.com/paper/2505.05019/) (2025)
- [Building Trust in Virtual Immunohistochemistry: Automated Assessment of Image Quality](https://mlsanity.com/paper/2511.04615/) (2025)
- [Assessment of Conditional Diffusion Model for Synthetic Histopathology Image Generation](https://mlsanity.com/paper/2608.03990/) (2026)

## Abstract (arXiv)

Although interest in synthetic medical data (SMD) for training and testing AI methods is growing, the absence of a standardized framework to evaluate its quality and applicability hinders its wider adoption. Here, we outline an evaluation framework designed to meet the unique requirements of medical applications, and introduce SMD Card, which can serve as comprehensive reports that accompany artificially generated datasets. This card provides a transparent and standardized framework for evaluating and reporting the quality of synthetic data, which can benefit SMD developers, users, and regulators, particularly for AI models using SMD in regulatory submissions.