mlsanityData to 14 Oct 2026

MAPLE: Metadata Augmented Private Language Evolution

arXiv 2603.19258 ↗First posted 26 Feb 2026cs.CL cs.AI cs.CRcited by 0

Not on the map. Why?

Abstract (arXiv)

Differentially private (DP) fine-tuning of large language models (LLMs) requires massive compute and full model access, which rules out state-of-the-art proprietary APIs for general users. Generating DP synthetic data offers a practical workaround. This approach also allows for transparent exploratory data analysis and arbitrary reuse across downstream tasks, sidestepping the rigid constraints of a model's parameter space. Private Evolution (PE) provides a promising API-based framework for generating this data, but its success relies heavily on initialization. If the private data distribution falls too far outside the foundation model's pre-training priors -- a common issue in highly specialized domain -- PE struggles to align with the target data. This misalignment causes poor convergence, degraded utility, and wasted API calls. To solve this initialization bottleneck, we introduce Metadata Augmented Private Language Evolution (MAPLE). MAPLE extracts DP tabular metadata and uses in-context learning to firmly ground the initial synthetic distribution in the target domain. Our evaluations on domain-specific text generation tasks show that MAPLE yields a strictly better privacy-utility trade-off, converges significantly faster, and sharply reduces API costs compared to baseline PE methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 42 of its 76 references that are in the map. First posted on arXiv 26 Feb 2026.

Builds on

42 of 76 references.

References · 42

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2019Advances and Open Problems in Federated Learningcited by 3,046
  8. 2016Deep Learning with Differential Privacycited by 2,728
  9. 2020Extracting Training Data from Large Language Modelscited by 2,189
  10. 2017Renyi Differential Privacycited by 650
  11. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  12. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
30 more
  1. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
  2. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 346
  3. 2013The Composition Theorem for Differential Privacycited by 297
  4. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 285
  5. 2017Collecting Telemetry Data Privatelycited by 149
  6. 2023Privacy Auditing with One (1) Training Runcited by 123
  7. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  8. 2022AIM: An Adaptive and Iterative Mechanism for Differentially Private Synthetic Datacited by 90
  9. 2023Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Modelscited by 83
  10. 2024Differentially Private Synthetic Data via Foundation Model APIs 2: Textcited by 76
  11. 2023Federated Learning of Gboard Language Models with Differential Privacycited by 66
  12. 2023Differentially Private Synthetic Data via Foundation Model APIs 1: Imagescited by 65
  13. 2023Harnessing large-language models to generate private synthetic textcited by 65
  14. 2023DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineercited by 57
  15. 2023SoK: Privacy-Preserving Data Synthesiscited by 38
  16. 2024Privacy-Preserving Instructions for Aligning Large Language Modelscited by 35
  17. 2024PrE-Text: Training Language Models on Private Federated Data in the Age of LLMscited by 29
  18. 2024Private prediction for large-scale synthetic text generationcited by 24
  19. 2024Tighter Privacy Auditing of DP-SGD in the Hidden State Threat Modelcited by 18
  20. 2025The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Textcited by 18
  21. 2025Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMscited by 16
  22. 2025VaultGemma: A Differentially Private Gemma Modelcited by 16
  23. 2025POPri: Private Federated Learning using Preference-Optimized Synthetic Datacited by 14
  24. 2025How to DP-fy Your Data: A Practical Guide to Generating Synthetic Data With Differential Privacycited by 13
  25. 2025Contrastive Private Data Synthesis via Weighted Multi-PLM Fusioncited by 11
  26. 2023Synthetic Query Generation for Privacy-Preserving Deep Retrieval Systems using Differentially Private Language Modelscited by 10
  27. 2024Private Synthetic Text Generation with Diffusion Modelscited by 10
  28. 2025Struct-Bench: A Benchmark for Differentially Private Structured Text Generationcited by 7
  29. 2025ACTG-ARL: Differentially Private Conditional Text Generation with RL-Boosted Controlcited by 3
  30. 2025Synthesizing and Adapting Error Correction Data for Mobile Large Language Model Applicationscited by 2

Built on by

Not cited yet.