mlsanityData to 14 Oct 2026

On Learning Representations for Tabular Data Distillation

arXiv 2501.13905 ↗First posted 23 Jan 2025cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Dataset distillation generates a small set of information-rich instances from a large dataset, resulting in reduced storage requirements, privacy or copyright risks, and computational costs for downstream modeling, though much of the research has focused on the image data modality. We study tabular data distillation, which brings in novel challenges such as the inherent feature heterogeneity and the common use of non-differentiable learning models (such as decision tree ensembles and nearest-neighbor predictors). To mitigate these challenges, we present textttTDColER, a tabular data distillation framework via column embeddings-based representation learning. To evaluate this framework, we also present a tabular data distillation benchmark, sf small TDBench. Based on an elaborate evaluation on sf small TDBench, resulting in 226,890 distilled datasets and 548,880 models trained on them, we demonstrate that textttTDColER is able to boost the distilled data quality of off-the-shelf distillation schemes by 0.5-143% across 7 different tabular learning models.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 42 of its 55 references that are in the map. First posted on arXiv 23 Jan 2025.

Builds on

42 of 55 references.

References · 42

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  2. 2017Graph Attention Networkscited by 8,923
  3. 2017Inductive Representation Learning on Large Graphscited by 6,320
  4. 2022Generative Adversarial Networkscited by 5,627
  5. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  6. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  7. 2018Experience Replay for Continual Learningcited by 1,095
  8. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  9. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 678
  10. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 676
  11. 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 533
  12. 2014OpenML: networked science in machine learningcited by 526
30 more
  1. 2020Dataset Condensation with Gradient Matchingcited by 516
  2. 2021Deep Neural Networks and Tabular Data: A Surveycited by 491
  3. 2022Dataset Distillation by Matching Training Trajectoriescited by 442
  4. 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
  5. 2021Dataset Condensation with Distribution Matchingcited by 365
  6. 2020Dataset Meta-Learning from Kernel Ridge-Regressioncited by 230
  7. 2020GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learningcited by 226
  8. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 223
  9. 2022On Embeddings for Numerical Features in Tabular Deep Learningcited by 186
  10. 2022DeepCore: A Comprehensive Library for Coreset Selection in Deep Learningcited by 147
  11. 2023Dataset Distillation: A Comprehensive Reviewcited by 127
  12. 2023Generalizing Dataset Distillation via Deep Generative Priorcited by 126
  13. 2023Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matchingcited by 118
  14. 2020Distilled One-Shot Federated Learningcited by 109
  15. 2019Soft-Label Dataset Distillation and Text Dataset Distillationcited by 107
  16. 2022Dataset Condensation with Contrastive Signalscited by 105
  17. 2021GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learningcited by 95
  18. 2016Entity Embeddings of Categorical Variablescited by 87
  19. 2022DC-BENCH: Dataset Condensation Benchmarkcited by 67
  20. 2020Federated Learning via Synthetic Datacited by 54
  21. 2022Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environmentscited by 49
  22. 2013Submodular Optimization with Submodular Cover and Submodular Knapsack Constraintscited by 41
  23. 2022Delving into Effective Gradient Matching for Dataset Condensationcited by 40
  24. 2020Sequential Deep Learning for Credit Risk Monitoring with Tabular Financial Datacited by 39
  25. 2021Distilled Replay: Overcoming Forgetting through Synthetic Samplescited by 36
  26. 2021Data Distillation for Text Classificationcited by 25
  27. 2021Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approachcited by 20
  28. 2024DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillationcited by 20
  29. 2020New Properties of the Data Distillation Method When Working With Tabular Datacited by 5
  30. 2023Embarassingly Simple Dataset Distillationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Language Model Representations for Efficient Few-Shot Tabular Classificationcited by 2
  2. 2026TabKD: Tabular Knowledge Distillation through Interaction Diversity of Learned Feature Binscited by 1
  3. 2026Context-Constrained Transfer Learning for Tabular Foundation Models via Data Distillationcited by 1
  4. 2026Adaptive Ensemble Selection for Noisy Labels on Tabular Datacited by 0