# New Properties of the Data Distillation Method When Working With Tabular Data

arXiv: https://arxiv.org/abs/2010.09839 · HTML: https://mlsanity.com/paper/2010.09839/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2020.

## Built on by (most cited first, 5 of 5)

- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [A Comprehensive Survey of Dataset Distillation](https://mlsanity.com/paper/2301.05603/) (2023)
- [Learning to Generate Synthetic Training Data using Gradient Matching and Implicit Differentiation](https://mlsanity.com/paper/2203.08559/) (2022)
- [On Learning Representations for Tabular Data Distillation](https://mlsanity.com/paper/2501.13905/) (2025)
- [Practical Knowledge Distillation: Using DNNs to Beat DNNs](https://mlsanity.com/paper/2302.12360/) (2023)

## Abstract (arXiv)

Data distillation is the problem of reducing the volume oftraining data while keeping only the necessary information. With thispaper, we deeper explore the new data distillation algorithm, previouslydesigned for image data. Our experiments with tabular data show thatthe model trained on distilled samples can outperform the model trainedon the original dataset. One of the problems of the considered algorithmis that produced data has poor generalization on models with differenthyperparameters. We show that using multiple architectures during distillation can help overcome this problem.