# Evaluating LLM Simulators as Differentially Private Data Generators

arXiv: https://arxiv.org/abs/2604.15461 · HTML: https://mlsanity.com/paper/2604.15461/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 9 references that are in the map.
- First posted on arXiv 16 Apr 2026.

## Builds on (most cited first, 5 of 5)

- [Winning the NIST Contest: A scalable and general approach to differentially private synthetic data](https://mlsanity.com/paper/2108.04978/) (2021)
- [Benchmarking Differentially Private Tabular Data Synthesis](https://mlsanity.com/paper/2504.14061/) (2025)
- [Synthetic Data Privacy Metrics](https://mlsanity.com/paper/2501.03941/) (2025)
- [The Importance of Being Discrete: Measuring the Impact of Discretization in End-to-End Differentially Private Synthetic Data](https://mlsanity.com/paper/2504.06923/) (2025)
- [PersonaLedger: Generating Realistic Financial Transactions with Persona Conditioned LLMs and Rule Grounded Feedback](https://mlsanity.com/paper/2601.03149/) (2026)

## Abstract (arXiv)

LLM-based simulators offer a promising path for generating complex synthetic data where traditional differentially private (DP) methods struggle with high-dimensional user profiles. But can LLMs faithfully reproduce statistical distributions from DP-protected inputs? We evaluate this using PersonaLedger, an agentic financial simulator, seeded with DP synthetic personas derived from real user statistics. We find that PersonaLedger achieves promising fraud detection utility (AUC 0.70 at epsilon=1) but exhibits significant distribution drift due to systematic LLM biases--learned priors overriding input statistics for temporal and demographic features. These failure modes must be addressed before LLM-based methods can handle the richer user representations where they might otherwise excel.