mlsanityData to 14 Oct 2026

Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimes

arXiv 2312.12112 ↗First posted 19 Dec 2023cs.LG cs.AIcited by 52

Not on the map. Why?

Abstract (arXiv)

Machine Learning (ML) in low-data settings remains an underappreciated yet crucial problem. Hence, data augmentation methods to increase the sample size of datasets needed for ML are key to unlocking the transformative potential of ML in data-deprived regions and domains. Unfortunately, the limited training set constrains traditional tabular synthetic data generators in their ability to generate a large and diverse augmented dataset needed for ML tasks. To address this challenge, we introduce CLLM, which leverages the prior knowledge of Large Language Models (LLMs) for data augmentation in the low-data regime. However, not all the data generated by LLMs will improve downstream utility, as for any generative model. Consequently, we introduce a principled curation mechanism, leveraging learning dynamics, coupled with confidence and uncertainty metrics, to obtain a high-quality dataset. Empirically, on multiple real-world datasets, we demonstrate the superior performance of CLLM in the low-data regime compared to conventional generators. Additionally, we provide insights into the LLM generation and curation mechanism, shedding light on the features that enable them to output high-quality augmented datasets.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). Builds on 50 of its 98 references that are in the map. First posted on arXiv 19 Dec 2023.

Builds on

50 of 98 references.

References · 50

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  6. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  7. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  8. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  9. 2018Datasheets for Datasetscited by 1,386
  10. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,323
  11. 2019A Comprehensive Survey on Transfer Learningcited by 1,160
  12. 2020Meta-Learning in Neural Networks: A Surveycited by 950
38 more
  1. 2017A Closer Look at Memorization in Deep Networkscited by 942
  2. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  3. 2023Large Language Models as Optimizerscited by 864
  4. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  5. 2014Certifying and removing disparate impactcited by 715
  6. 2019Modeling Tabular data using Conditional GANcited by 699
  7. 2020Self-supervised Learning: Generative or Contrastivecited by 694
  8. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  9. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  10. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  11. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 566
  12. 2021Tabular Data: Deep Learning is Not All You Needcited by 535
  13. 2014OpenML: networked science in machine learningcited by 526
  14. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 432
  15. 2018Assessing Generative Models via Precision and Recallcited by 406
  16. 2023Larger language models do in-context learning differentlycited by 394
  17. 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 377
  18. 2017Real-valued (Medical) Time Series Generation with Recurrent Conditional GANscited by 333
  19. 2022Language Models are Realistic Tabular Data Generatorscited by 294
  20. 2020A Brief Review of Domain Adaptationcited by 288
  21. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 277
  22. 2023Data-centric Artificial Intelligence: A Surveycited by 203
  23. 2023Large Language Models as General Pattern Machinescited by 197
  24. 2016Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learningcited by 195
  25. 2019A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cyclecited by 193
  26. 2017A Convex Framework for Fair Regressioncited by 148
  27. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 109
  28. 2022STaSy: Score-based Tabular data Synthesiscited by 86
  29. 2023Synthcity: facilitating innovative use cases of synthetic data in different data modalitiescited by 76
  30. 2019Data Augmentation Using GANscited by 58
  31. 2022Transfer Learning with Deep Tabular Modelscited by 52
  32. 2022Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learningcited by 36
  33. 2021What can Data-Centric AI Learn from Data and ML Engineering?cited by 24
  34. 2022Data-IQ: Characterizing subgroups with heterogeneous outcomes in tabular datacited by 22
  35. 2018On the Relationship between Data Efficiency and Error for Uncertainty Samplingcited by 12
  36. 2023Generative Adversarial Networks for Data Augmentationcited by 7
  37. 2023TRIAGE: Characterizing and auditing training data for improved regressioncited by 6
  38. 2023Enabling tabular deep learning when d gg n with an auxiliary knowledge graphcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 18; 2025: 17; 2026: 17.

52 papers.

Sort:

By area:

  1. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  2. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 159
  3. 2024Large Language Models to Enhance Bayesian Optimizationcited by 135
  4. 2024A Comprehensive Survey on Data Augmentationcited by 40
  5. 2025A Comprehensive Survey of Synthetic Tabular Data Generationcited by 28
  6. 2024TabEBM: A Tabular Data Augmentation Method with Distinct Class-Specific Energy-Based Modelscited by 16
  7. 2024EPIC: Effective Prompting for Imbalanced-Class Data Synthesis in Tabular Data Classification via Large Language Modelscited by 14
  8. 2024CTSyn: A Foundation Model for Cross Tabular Data Generationcited by 14
  9. 2025A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMscited by 13
  10. 2025Towards Objective Fine-tuning: How LLMs' Prior Knowledge Causes Potential Poor Calibration?cited by 11
  11. 2025Few-shot LLM Synthetic Data with Distribution Matchingcited by 10
  12. 2025TabGen-ICL: Residual-Aware In-Context Example Selection for Tabular Data Generationcited by 10
38 more
  1. 2024Generating Realistic Tabular Data with Large Language Modelscited by 9
  2. 2024Matchmaker: Self-Improving Large Language Model Programs for Schema Matchingcited by 9
  3. 2024Context-Aware Testing: A New Paradigm for Model Testing with Large Language Modelscited by 8
  4. 2025Not All Features Deserve Attention: Graph-Guided Dependency Learning for Tabular Data Generation with Language Modelscited by 8
  5. 2024MALLM-GAN: Multi-Agent Large Language Model as Generative Adversarial Network for Synthesizing Tabular Datacited by 7
  6. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Datacited by 6
  7. 2024TabMDA: Tabular Manifold Data Augmentation for Any Classifier using Transformers with In-context Subsettingcited by 5
  8. 2025TabRep: Training Tabular Diffusion Models with a Simple and Effective Continuous Representationcited by 5
  9. 2024A text-to-tabular approach to generate synthetic patient data using LLMscited by 4
  10. 2025Tabby: A Language Model Architecture for Tabular and Structured Data Synthesiscited by 4
  11. 2025Large Language Models for Imbalanced Classification: Diversity makes the differencecited by 4
  12. 2026DARA: Few-shot Budget Allocation in Online Advertising via In-Context Decision Making with RL-Finetuned LLMscited by 4
  13. 2024iTBLS: A Dataset of Interactive Conversations Over Tabular Informationcited by 3
  14. 2024Generating Tables from the Parametric Knowledge of Language Modelscited by 3
  15. 2024SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipecited by 3
  16. 2025LLM Empowered Prototype Learning for Zero and Few-Shot Tasks on Tabular Datacited by 2
  17. 2025Robustness is Important: Limitations of LLMs for Data Fittingcited by 2
  18. 2025Large Language Models as Attribution Regularizers for Efficient Model Trainingcited by 1
  19. 2025Assessing Generative Models for Structured Datacited by 1
  20. 2025In-Context Bias Propagation in LLM-Based Tabular Data Generationcited by 1
  21. 2025FairTabGen: High-Fidelity and Fair Synthetic Health Data Generation from Limited Samplescited by 1
  22. 2026Team, Then Trim: An Assembly-Line LLM Framework for High-Quality Tabular Data Generationcited by 1
  23. 2026ReTabSyn: Realistic Tabular Data Synthesis via Reinforcement Learningcited by 1
  24. 2026Task Expansion and Cross Refinement for Open-World Conditional Modelingcited by 1
  25. 2026Noise-Response Calibration: A Causal Intervention Protocol for LLM-Judgescited by 1
  26. 2026Adversarial Arena: Crowdsourcing Data Generation through Interactive Competitioncited by 1
  27. 2026SAGE: Sparse Adaptive Guidance for Dependency-Aware Tabular Data Generationcited by 1
  28. 2026Active Tabular Augmentation via Policy-Guided Diffusion Inpaintingcited by 1
  29. 2024Unlocking Historical Clinical Trial Data with ALIGN: A Compositional Large Language Model System for Medical Codingcited by 0
  30. 2024Improving Equity in Health Modeling with GPT4-Turbo Generated Synthetic Data: A Comparative Studycited by 0
  31. 2025AI for pRedicting Exacerbations in KIDs with aSthma (AIRE-KIDS)cited by 0
  32. 2025Exploring the Heterogeneity of Tabular Data: A Diversity-aware Data Generator via LLMscited by 0
  33. 2026LLM-Driven Performance-Space Augmentation for Meta-Learning-Based Algorithm Selectioncited by 0
  34. 2026Using Zero-Shot LLM-Generated Survey Data for Geographically Explicit Population Synthesiscited by 0
  35. 2026UniD3: A Knowledge Graph-Enhanced RAG Framework for Drug-Disease Discovery and Reasoningcited by 0
  36. 2026Cross-Domain Feature Expansion for Tabular Medical Data via Knowledge Graphs Injectioncited by 0
  37. 2026Generative AI and Federated Learning for Intrusion Detection Systems: A Surveycited by 0
  38. 2026Creation begins with understanding: LLMs as strategy designers for privacy-preserving tabular data synthesiscited by 0

50 most cited of 52 All 52 (JSON)