A Bias-Variance Decomposition for Ensembles over Multiple Synthetic Datasets
Not on the map. Why?
Abstract (arXiv)
Recent studies have highlighted the benefits of generating multiple synthetic datasets for supervised learning, from increased accuracy to more effective model selection and uncertainty estimation. These benefits have clear empirical support, but the theoretical understanding of them is currently very light. We seek to increase the theoretical understanding by deriving bias-variance decompositions for several settings of using multiple synthetic datasets, including differentially private synthetic data. Our theory yields a simple rule of thumb to select the appropriate number of synthetic datasets in the case of mean-squared error and Brier score. We investigate how our theory works in practice with several real datasets, downstream predictors and error metrics. As our theory predicts, multiple synthetic datasets often improve accuracy, while a single large synthetic dataset gives at best minimal improvement, showing that our insights are practically relevant.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 33 of its 70 references that are in the map. First posted on arXiv 6 Feb 2024.
Builds on
33 of 70 references.
References · 33
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
- 2019Modeling Tabular data using Conditional GANcited by 695
- 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
- 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 443
- 2020Reliable Fidelity and Diversity Metrics for Generative Modelscited by 425
- 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 373
- 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
- 2017Data Augmentation Generative Adversarial Networkscited by 278
- 2021How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Modelscited by 193
- 2020GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generatorscited by 117
- 2021DECAF: Generating Fair Synthetic Data Using Causally-Aware Generative Networkscited by 109
- 2020Synthetic Data -- Anonymisation Groundhog Daycited by 105
21 more
- 2021Winning the NIST Contest: A scalable and general approach to differentially private synthetic datacited by 101
- 2022AIM: An Adaptive and Iterative Mechanism for Differentially Private Synthetic Datacited by 88
- 2020DP-MERF: Differentially Private Mean Embeddings with Random Features for Practical Privacy-Preserving Data Generationcited by 78
- 2019Graphical-model based estimation and inference for differential privacycited by 73
- 2023A Unified Theory of Diversity in Ensemble Learningcited by 71
- 2023Membership Inference Attacks against Synthetic Data through Overfitting Detectioncited by 66
- 2021Iterative Methods for Private Synthetic Data: Unifying Framework and New Methodscited by 58
- 2021Differentially Private Query Release Through Adaptive Projectioncited by 50
- 2023Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Datacited by 31
- 2023Synthetic data, real errors: how (not) to publish and use synthetic datacited by 29
- 2023Achilles' Heels: Vulnerable Record Identification in Synthetic Data Publishingcited by 28
- 2021Conditional Synthetic Data Generation for Robust Machine Learning Applications with Limited Pandemic Datacited by 18
- 2016Differentially Private Variational Inference for Non-conjugate Modelscited by 16
- 2018Multiple Imputation: A Review of Practical and Theoretical Findingscited by 14
- 2022Noise-Aware Statistical Inference with Differentially Private Synthetic Datacited by 9
- 2016Ensembles of Generative Adversarial Networkscited by 8
- 2019Privacy-preserving data sharing via probabilistic modellingcited by 7
- 2022Proper losses for discrete generative modelscited by 4
- 2020Ensembles of Generative Adversarial Networks for Disconnected Datacited by 3
- 2023On Consistent Bayesian Inference from Synthetic Datacited by 2
- 2020Constrained Generative Adversarial Network Ensembles for Sharable Synthetic Data Generationcited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.