# Really Useful Synthetic Data -- A Framework to Evaluate the Quality of Differentially Private Synthetic Data

arXiv: https://arxiv.org/abs/2004.07740 · HTML: https://mlsanity.com/paper/2004.07740/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2020.

## Built on by (most cited first, 11 of 11)

- [Can I trust my fake data -- A comprehensive quality assessment framework for synthetic tabular data in healthcare](https://mlsanity.com/paper/2401.13716/) (2024)
- [Graphical vs. Deep Generative Models: Measuring the Impact of Differentially Private Mechanisms and Budgets on Utility](https://mlsanity.com/paper/2305.10994/) (2023)
- [KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Server](https://mlsanity.com/paper/2410.05725/) (2024)
- [Struct-Bench: A Benchmark for Differentially Private Structured Text Generation](https://mlsanity.com/paper/2509.10696/) (2025)
- [Generating tabular datasets under differential privacy](https://mlsanity.com/paper/2308.14784/) (2023)
- [Does Differentially Private Synthetic Data Lead to Synthetic Discoveries?](https://mlsanity.com/paper/2403.13612/) (2024)
- [Spending Privacy Budget Fairly and Wisely](https://mlsanity.com/paper/2204.12903/) (2022)
- [Privacy-Preserving Fair Synthetic Tabular Data](https://mlsanity.com/paper/2503.02968/) (2025)
- [Quantitative Auditing of AI Fairness with Differentially Private Synthetic Data](https://mlsanity.com/paper/2504.21634/) (2025)
- [Permissioned Blockchain-based Framework for Ranking Synthetic Data Generators](https://mlsanity.com/paper/2405.07196/) (2024)
- [NFDI4Health workflow and service for synthetic data generation, assessment and risk management](https://mlsanity.com/paper/2408.04478/) (2024)

## Abstract (arXiv)

Recent advances in generating synthetic data that allow to add principled ways of protecting privacy -- such as Differential Privacy -- are a crucial step in sharing statistical information in a privacy preserving way. But while the focus has been on privacy guarantees, the resulting private synthetic data is only useful if it still carries statistical information from the original data. To further optimise the inherent trade-off between data privacy and data quality, it is necessary to think closely about the latter. What is it that data analysts want? Acknowledging that data quality is a subjective concept, we develop a framework to evaluate the quality of differentially private synthetic data from an applied researcher's perspective. Data quality can be measured along two dimensions. First, quality of synthetic data can be evaluated against training data or against an underlying population. Second, the quality of synthetic data depends on general similarity of distributions or specific tasks such as inference or prediction. It is clear that accommodating all goals at once is a formidable challenge. We invite the academic community to jointly advance the privacy-quality frontier.