# A Comprehensive Benchmark of Machine and Deep Learning Across Diverse Tabular Datasets

arXiv: https://arxiv.org/abs/2408.14817 · HTML: https://mlsanity.com/paper/2408.14817/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 48 references that are in the map.
- First posted on arXiv 27 Aug 2024.

## Builds on (most cited first, 16 of 16)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [Tabular Data: Deep Learning is Not All You Need](https://mlsanity.com/paper/2106.03253/) (2021)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020)
- [SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training](https://mlsanity.com/paper/2106.01342/) (2021)
- [When Do Neural Nets Outperform Boosted Trees on Tabular Data?](https://mlsanity.com/paper/2305.02997/) (2023)
- [Well-tuned Simple Nets Excel on Tabular Datasets](https://mlsanity.com/paper/2106.11189/) (2021)
- [A Comparative Analysis of XGBoost](https://mlsanity.com/paper/1911.01914/) (2019)
- [Beyond One-hot Encoding: lower dimensional target embedding](https://mlsanity.com/paper/1806.10805/) (2018)
- [A Closer Look at Deep Learning Methods on Tabular Datasets](https://mlsanity.com/paper/2407.00956/) (2024)
- [Generalization bounds for deep learning](https://mlsanity.com/paper/2012.04115/) (2020)
- [SubStrat: A Subset-Based Strategy for Faster AutoML](https://mlsanity.com/paper/2206.03070/) (2022)
- [A Novel Meta Learning Framework for Feature Selection using Data Synthesis and Fuzzy Similarity](https://mlsanity.com/paper/2005.09856/) (2020)

## Built on by (most cited first, 15 of 15)

- [Beyond IID: How General Are Tabular Foundation Models, Really?](https://mlsanity.com/paper/2606.30410/) (2026)
- [TabSTAR: A Tabular Foundation Model for Tabular Data with Text Fields](https://mlsanity.com/paper/2505.18125/) (2025)
- [Modern Neural Networks for Small Tabular Datasets: The New Default for Field-Scale Digital Soil Mapping?](https://mlsanity.com/paper/2508.09888/) (2025)
- [LimeSoDa: A Dataset Collection for Benchmarking of Machine Learning Regressors in Digital Soil Mapping](https://mlsanity.com/paper/2502.20139/) (2025)
- [On the retraining frequency of global models in retail demand forecasting](https://mlsanity.com/paper/2505.00356/) (2025)
- [MultiTab: A Comprehensive Benchmark Suite for Multi-Dimensional Evaluation in Tabular Domains](https://mlsanity.com/paper/2505.14312/) (2025)
- [TabPrep: Closing the Feature Engineering Gap in Tabular Benchmarks](https://mlsanity.com/paper/2606.02384/) (2026)
- [Tuberculosis Screening from Cough Audio: Baseline Models, Clinical Variables, and Uncertainty Quantification](https://mlsanity.com/paper/2601.07969/) (2026)
- [Explaining Tabular Foundation Model Differences Through Meta-Features](https://mlsanity.com/paper/2605.28418/) (2026)
- [Cold-Start Active Preference Learning in Socio-Economic Domains](https://mlsanity.com/paper/2508.05090/) (2025)
- [Near Optimal Inference for the Best-Performing Algorithm](https://mlsanity.com/paper/2508.05173/) (2025)
- [Who is the Winning Algorithm? Rank Aggregation for Comparative Studies](https://mlsanity.com/paper/2601.01664/) (2026)
- [ZKBoost: Zero-Knowledge Verifiable Training for XGBoost](https://mlsanity.com/paper/2602.04113/) (2026)
- [DualFlexKAN: Dual-stage Kolmogorov-Arnold Networks with Independent Function Control](https://mlsanity.com/paper/2603.08583/) (2026)
- [Crossing the Validation Crisis: Cross-Validation Reduces Benchmarking Variance Surprisingly Well](https://mlsanity.com/paper/2606.12552/) (2026)

## Abstract (arXiv)

The analysis of tabular datasets is highly prevalent both in scientific research and real-world applications of Machine Learning (ML). Unlike many other ML tasks, Deep Learning (DL) models often do not outperform traditional methods in this area. Previous comparative benchmarks have shown that DL performance is frequently equivalent or even inferior to models such as Gradient Boosting Machines (GBMs). In this study, we introduce a comprehensive benchmark aimed at better characterizing the types of datasets where DL models excel. Although several important benchmarks for tabular datasets already exist, our contribution lies in the variety and depth of our comparison: we evaluate 111 datasets with 20 different models, including both regression and classification tasks. These datasets vary in scale and include both those with and without categorical variables. Importantly, our benchmark contains a sufficient number of datasets where DL models perform best, allowing for a thorough analysis of the conditions under which DL models excel. Building on the results of this benchmark, we train a model that predicts scenarios where DL models outperform alternative methods with 86.1% accuracy (AUC 0.78). We present insights derived from this characterization and compare these findings to previous benchmarks.