mlsanityData to 7 Oct 2026

Privately generating tabular data using language models

arXiv 2306.04803 ↗First posted 7 Jun 2023cs.LG cs.CL cs.CRcited by 5

Not on the map. Why?

Abstract (arXiv)

Privately generating synthetic data from a table is an important brick of a privacy-first world. We propose and investigate a simple approach of treating each row in a table as a sentence and training a language model with differential privacy. We show this approach obtains competitive results in modelling tabular data across multiple datasets, even at small scales that favor alternative methods based on marginal distributions.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 23 of its 33 references that are in the map. First posted on arXiv 7 Jun 2023.

Builds on

23 of 33 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  7. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  8. 2023MusicLM: Generating Music From Textcited by 442
  9. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  10. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
  11. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 233
  12. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 218
11 more
  1. 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 199
  2. 2022CM3: A Causal Masked Multimodal Model of the Internetcited by 142
  3. 2021Large-Scale Differentially Private BERTcited by 117
  4. 2021Winning the NIST Contest: A scalable and general approach to differentially private synthetic datacited by 101
  5. 2022AIM: An Adaptive and Iterative Mechanism for Differentially Private Synthetic Datacited by 88
  6. 2021Benchmarking Differentially Private Synthetic Data Generation Algorithmscited by 80
  7. 2019Graphical-model based estimation and inference for differential privacycited by 73
  8. 2021Iterative Methods for Private Synthetic Data: Unifying Framework and New Methodscited by 58
  9. 2022TAN Without a Burn: Scaling Laws of DP-SGDcited by 51
  10. 2021Composable Generative Modelscited by 3
  11. 2022Generative Modeling of Complex Datacited by 3

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 3.

5 papers.

  1. 2024Differentially Private Tabular Data Synthesis using Large Language Modelscited by 16
  2. 2025Synthetic Tabular Data: Methods, Attacks and Defensescited by 8
  3. 2025Is API Access to LLMs Useful for Generating Private Synthetic Tabular Data?cited by 5
  4. 2024Qrlew: Rewriting SQL into Differentially Private SQLcited by 1
  5. 2025Beyond One-Size-Fits-All: Neural Networks for Differentially Private Tabular Data Synthesiscited by 0