mlsanityData to 7 Oct 2026

Towards High Supervised Learning Utility Training Data Generation: Data Pruning and Column Reordering

arXiv 2507.10088 ↗First posted 14 Jul 2025cs.LG stat.MLcited by 0

On the map: Synthetic data and self-improvement for LLMs → Privacy-preserving tabular data synthesis

Abstract (arXiv)

Tabular data synthesis for supervised learning ('SL') model training is gaining popularity in industries such as healthcare, finance, and retail. Despite the progress made in tabular data generators, models trained with synthetic data often underperform compared to those trained with original data. This low SL utility of synthetic data stems from class imbalance exaggeration and SL data relationship overlooked by tabular generator. To address these challenges, we draw inspirations from techniques in emerging data-centric artificial intelligence and elucidate Pruning and ReOrdering ('PRRO'), a novel pipeline that integrates data-centric techniques into tabular data synthesis. PRRO incorporates data pruning to guide the table generator towards observations with high signal-to-noise ratio, ensuring that the class distribution of synthetic data closely matches that of the original data. Besides, PRRO employs a column reordering algorithm to align the data modeling structure of generators with that of SL models. These two modules enable PRRO to optimize SL utility of synthetic data. Empirical experiments on 22 public datasets show that synthetic data generated using PRRO enhances predictive performance compared to data generated without PRRO. Specifically, synthetic replacement of original data yields an average improvement of 26.74% and up to 871.46% improvement using PRRO, while synthetic appendant to original data results with PRRO-generated data results in an average improvement of 6.13% and up to 200.32%. Furthermore, experiments on six highly imbalanced datasets show that PRRO enables the generator to produce synthetic data with a class distribution that resembles the original data more closely, achieving a similarity improvement of 43%. Through PRRO, we foster a seamless integration of data synthesis to subsequent SL prediction, promoting quality and accessible data analysis.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 27 of its 69 references that are in the map. First posted on arXiv 14 Jul 2025.

Builds on

27 of 69 references.

References · 27

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  4. 2019Modeling Tabular data using Conditional GANcited by 695
  5. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 673
  6. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  7. 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 411
  8. 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 373
  9. 2022Language Models are Realistic Tabular Data Generatorscited by 290
  10. 2023Generating Images with Multimodal Language Modelscited by 259
  11. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
  12. 2014A Tutorial on Principal Component Analysiscited by 139
15 more
  1. 2023REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformerscited by 93
  2. 2023Split and Merge: Aligning Position Biases in LLM-based Evaluatorscited by 71
  3. 2024Eliminating Position Bias of Language Models: A Mechanistic Approachcited by 61
  4. 2024Generative Artificial Intelligence: A Systematic Review and Applicationscited by 37
  5. 2023Real-Fake: Effective Training Data Synthesis Through Distribution Matchingcited by 35
  6. 2019Signal-to-Noise Ratio: A Robust Distance Metric for Deep Metric Learningcited by 32
  7. 2022To SMOTE, or not to SMOTE?cited by 14
  8. 2023Interpreting the Curse of Dimensionality from Distance Concentration and Manifold Effectcited by 14
  9. 2024Graph Neural Networks for Tabular Data Learning: A Survey with Taxonomy and Directionscited by 13
  10. 2023SYNAuG: Exploiting Synthetic Data for Data Imbalance Problemscited by 12
  11. 2024A Survey of Data Synthesis Approachescited by 8
  12. 2020Privacy for All: Demystify Vulnerability Disparity of Differential Privacy against Membership Inference Attackcited by 4
  13. 2022Stop Oversampling for Class Imbalance Learning: A Critical Reviewcited by 4
  14. 2023Generalized Oversampling for Learning from Imbalanced datasets and Associated Theorycited by 4
  15. 2024DEREC-SIMPRO: unlock Language Model benefits to advance Synthesis in Data Clean Roomcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Comprehensive Survey of Synthetic Tabular Data Generation
  2. 2026Self-Reinforcing Controllable Synthesis of Rare Relational Data via Bayesian Calibration
  3. 2024EPIC: Effective Prompting for Imbalanced-Class Data Synthesis in Tabular Data Classification via Large Language Models
  4. 2025FASTGEN: Fast and Cost-Effective Synthetic Tabular Data Generation with LLMs
  5. 2024A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data
  6. 2025Exploring the Heterogeneity of Tabular Data: A Diversity-aware Data Generator via LLMs
  7. 2025Risk In Context: Benchmarking Privacy Leakage of Foundation Models in Synthetic Tabular Data Generation
  8. 2026Self-Improving Tabular Language Models via Iterative Reward-Guided Post-Training