mlsanityData to 7 Oct 2026

A Multi-Faceted Evaluation Framework for Assessing Synthetic Data Generated by Large Language Models

arXiv 2404.14445 ↗First posted 20 Apr 2024cs.LG cs.AI cs.CLcited by 8

On the map: Synthetic data and self-improvement for LLMs → Privacy-preserving tabular data synthesis

Abstract (arXiv)

The rapid advancements in generative AI and large language models (LLMs) have opened up new avenues for producing synthetic data, particularly in the realm of structured tabular formats, such as product reviews. Despite the potential benefits, concerns regarding privacy leakage have surfaced, especially when personal information is utilized in the training datasets. In addition, there is an absence of a comprehensive evaluation framework capable of quantitatively measuring the quality of the generated synthetic data and their utility for downstream tasks. In response to this gap, we introduce SynEval, an open-source evaluation framework designed to assess the fidelity, utility, and privacy preservation of synthetically generated tabular data via a suite of diverse evaluation metrics. We validate the efficacy of our proposed framework - SynEval - by applying it to synthetic product review data generated by three state-of-the-art LLMs: ChatGPT, Claude, and Llama. Our experimental findings illuminate the trade-offs between various evaluation metrics in the context of synthetic data generation. Furthermore, SynEval stands as a critical instrument for researchers and practitioners engaged with synthetic tabular data,, empowering them to judiciously determine the suitability of the generated data for their specific applications, with an emphasis on upholding user privacy.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 13 of its 37 references that are in the map. First posted on arXiv 20 Apr 2024.

Builds on

13 of 37 references.

References · 13

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  4. 2021Membership Inference Attacks From First Principlescited by 839
  5. 2019Modeling Tabular data using Conditional GANcited by 695
  6. 2024Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoderscited by 213
  7. 2023Does Synthetic Data Generation of LLMs Help Clinical Text Mining?cited by 151
  8. 2015Training generative neural networks via Maximum Mean Discrepancy optimizationcited by 149
  9. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitationscited by 96
  10. 2017Adversarial Feature Matching for Text Generationcited by 58
  11. 2020A Tutorial on Learning With Bayesian Networkscited by 56
  12. 2017CVAE-GAN: Fine-Grained Image Generation through Asymmetric Trainingcited by 42
1 more
  1. 2023A Comparative Study of Open-Source Large Language Models, GPT-4 and Claude 2: Multiple-Choice Test Taking in Nephrologycited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 3.

8 papers.

  1. 2025Struct-Bench: A Benchmark for Differentially Private Structured Text Generationcited by 7
  2. 2025GReaTER: Generate Realistic Tabular data after data Enhancement and Reductioncited by 3
  3. 2025Can Large Language Models Generate Effective Datasets for Emotion Recognition in Conversations?cited by 2
  4. 2025Dataset Creation for Visual Entailment using Generative AIcited by 0
  5. 2025RL-Finetuned LLMs for Privacy-Preserving Synthetic Rewritingcited by 0
  6. 2026SynQP: A Framework and Metrics for Evaluating the Quality and Privacy Risk of Synthetic Datacited by 0
  7. 2026NodeSynth: Socially Aligned Synthetic Data for AI Evaluationcited by 0
  8. 2026Beyond the Stitching Assumption: A Unified Framework for Multimodal Synthetic Data Evaluation via Semantic Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Privacy-Preserving Synthetic Review Generation with Diverse Writing Styles Using LLMs
  2. 2023Boosting Data Analytics With Synthetic Volume Expansion
  3. 2025Risk In Context: Benchmarking Privacy Leakage of Foundation Models in Synthetic Tabular Data Generation
  4. 2025A Comprehensive Survey of Synthetic Tabular Data Generation
  5. 2024Best Practices and Lessons Learned on Synthetic Data
  6. 2024Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data
  7. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data
  8. 2025SynLLM: A Comparative Analysis of Large Language Models for Medical Tabular Synthetic Data Generation via Prompt Engineering