mlsanityData to 7 Oct 2026

Language Models are Realistic Tabular Data Generators

arXiv 2210.06280 ↗First posted 12 Oct 2022cs.LGcited by 290

Not on the map. Why?

Cited by 32 in Synthetic data and self-improvement for LLMs, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Tabular data is among the oldest and most ubiquitous forms of data. However, the generation of synthetic samples with the original data's characteristics remains a significant challenge for tabular data. While many generative models from the computer vision domain, such as variational autoencoders or generative adversarial networks, have been adapted for tabular data generation, less research has been directed towards recent transformer-based large language models (LLMs), which are also generative in nature. To this end, we propose GReaT (Generation of Realistic Tabular data), which exploits an auto-regressive generative LLM to sample synthetic and yet highly realistic tabular data. Furthermore, GReaT can model tabular data distributions by conditioning on any subset of features; the remaining features are sampled without additional overhead. We demonstrate the effectiveness of the proposed approach in a series of experiments that quantify the validity and quality of the produced data samples from multiple angles. We find that GReaT maintains state-of-the-art performance across numerous real-world and synthetic data sets with heterogeneous feature types coming in various sizes.

Key facts. Cited by 290 papers in the map (data through 7 Oct 2026). Builds on 33 of its 59 references that are in the map. First posted on arXiv 12 Oct 2022.

Builds on

33 of 59 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  8. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  9. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  10. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  11. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  12. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
21 more
  1. 2014Conditional Generative Adversarial Netscited by 2,152
  2. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  3. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  4. 2019Modeling Tabular data using Conditional GANcited by 695
  5. 2019TabNet: Attentive Interpretable Tabular Learningcited by 640
  6. 2021Deep Neural Networks and Tabular Data: A Surveycited by 489
  7. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
  8. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 310
  9. 2017Generating Multi-label Discrete Patient Records using Generative Adversarial Networkscited by 197
  10. 2018Data Synthesis based on Generative Adversarial Networkscited by 184
  11. 2019Do NLP Models Know Numbers? Probing Numeracy in Embeddingscited by 157
  12. 2019Learning Model-Agnostic Counterfactual Explanations for Tabular Datacited by 146
  13. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  14. 2021Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learningcited by 112
  15. 2020Tabular Transformers for Modeling Multivariate Time Seriescited by 77
  16. 2020VAEM: a Deep Generative Model for Heterogeneous Mixed Type Datacited by 66
  17. 2021Copula Flows for Synthetic Data Generationcited by 35
  18. 2017Adversarial Generation of Natural Languagecited by 33
  19. 2018Airline Passenger Name Record Generation using Generative Adversarial Networkscited by 19
  20. 2020Google Dataset Search by the Numberscited by 17
  21. 2021Synthesising Multi-Modal Minority Samples for Tabular Datacited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 38; 2024: 98; 2025: 88; 2026: 64.

290 papers.

Sort:

By area:

  1. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 273
  2. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 230
  3. 2023In-Context Unlearning: Language Models as Few Shot Unlearnerscited by 214
  4. 2023Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Spacecited by 183
  5. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
  6. 2023How Can Recommender Systems Benefit from Large Language Models: A Surveycited by 141
  7. 2023GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoningcited by 107
  8. 2024Best Practices and Lessons Learned on Synthetic Datacited by 104
  9. 2024Why Tabular Foundation Models Should Be a Research Prioritycited by 103
  10. 2023REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformerscited by 93
  11. 2023ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendationcited by 76
  12. 2023Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based Preferencescited by 66
38 more
  1. 2023Generating and Imputing Tabular Data via Diffusion and Flow-based Gradient-Boosted Treescited by 60
  2. 2024TabDiff: a Mixed-type Diffusion Model for Tabular Data Generationcited by 60
  3. 2024Making Pre-trained Language Models Great on Tabular Predictioncited by 56
  4. 2023Generative Table Pre-training Empowers Models for Tabular Predictioncited by 55
  5. 2023ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Modelscited by 54
  6. 2023Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimescited by 52
  7. 2024Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspectivecited by 51
  8. 2024SoK: Watermarking for AI-Generated Contentcited by 43
  9. 2023TabuLa: Harnessing Language Models for Tabular Data Synthesiscited by 42
  10. 2023Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matchingcited by 40
  11. 2023TabMT: Generating tabular data with masked transformerscited by 36
  12. 2024A Synthetic Dataset for Personal Attribute Inferencecited by 35
  13. 2024Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Modelscited by 34
  14. 2024Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendationcited by 33
  15. 2025CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning taskscited by 30
  16. 2023TABLET: Learning From Instructions For Tabular Datacited by 28
  17. 2023ClickPrompt: CTR Models are Strong Prompt Generators for Adapting Language Models to CTR Predictioncited by 27
  18. 2025A Comprehensive Survey of Synthetic Tabular Data Generationcited by 27
  19. 2024TabPFGen -- Tabular Data Generation with TabPFNcited by 26
  20. 2024Large Language Model Distilling Medication Recommendation Modelcited by 25
  21. 2023MediTab: Scaling Medical Tabular Data Predictors via Data Consolidation, Enrichment, and Refinementcited by 24
  22. 2024DrHouse: An LLM-empowered Diagnostic Reasoning System through Harnessing Outcomes from Sensor Data and Expert Knowledgecited by 24
  23. 2023Towards Cross-Table Masked Pretraining for Web Data Miningcited by 21
  24. 2023Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Modelscited by 20
  25. 2024Enhancing Recommendation Diversity by Re-ranking with Large Language Modelscited by 19
  26. 2024Elephants Never Forget: Testing Language Models for Memorization of Tabular Datacited by 19
  27. 2024Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolutioncited by 19
  28. 2023User Modeling in the Era of Large Language Models: Current Research and Future Directionscited by 18
  29. 2024A Survey on Deep Tabular Learningcited by 18
  30. 2025Beyond the convexity assumption: Realistic tabular data generation under quantifier-free real linear constraintscited by 18
  31. 2023TarGEN: Targeted Data Generation with Large Language Modelscited by 17
  32. 2024Systematic Assessment of Tabular Data Synthesiscited by 17
  33. 2023Continuous Diffusion for Mixed-Type Tabular Datacited by 16
  34. 2024Generative Text Steganography with Large Language Modelcited by 16
  35. 2024Differentially Private Tabular Data Synthesis using Large Language Modelscited by 16
  36. 2024Contextualized Sequence Likelihood: Enhanced Confidence Scores for Natural Language Generationcited by 16
  37. 2024HARMONIC: Harnessing LLMs for Tabular Data Synthesis and Privacy Protectioncited by 16
  38. 2024Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendationcited by 16

50 most cited of 290 All 290 (JSON)