# TabRepo: A Large Scale Repository of Tabular Model Evaluations and its AutoML Applications

arXiv: https://arxiv.org/abs/2311.02971 · HTML: https://mlsanity.com/paper/2311.02971/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 58 references that are in the map.
- First posted on arXiv 6 Nov 2023.

## Builds on (most cited first, 29 of 29)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [AutoML: A Survey of the State-of-the-Art](https://mlsanity.com/paper/1908.00709/) (2019)
- [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020)
- [NAS-Bench-101: Towards Reproducible Neural Architecture Search](https://mlsanity.com/paper/1902.09635/) (2019)
- [Multimodal datasets: misogyny, pornography, and malignant stereotypes](https://mlsanity.com/paper/2110.01963/) (2021)
- [When Do Neural Nets Outperform Boosted Trees on Tabular Data?](https://mlsanity.com/paper/2305.02997/) (2023)
- [Why do tree-based models still outperform deep learning on tabular data?](https://mlsanity.com/paper/2207.08815/) (2022)
- [Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning](https://mlsanity.com/paper/2007.04074/) (2020)
- [HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPO](https://mlsanity.com/paper/2109.06716/) (2021)
- [FLAML: A Fast and Lightweight AutoML Library](https://mlsanity.com/paper/1911.04706/) (2019)
- [An Open Source AutoML Benchmark](https://mlsanity.com/paper/1907.00909/) (2019)
- [XTab: Cross-table Pretraining for Tabular Transformers](https://mlsanity.com/paper/2305.06090/) (2023)
- [TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023](https://mlsanity.com/paper/2307.14338/) (2023)
- [NAS evaluation is frustratingly hard](https://mlsanity.com/paper/1912.12522/) (2019)
- [Auto-PyTorch Tabular: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL](https://mlsanity.com/paper/2006.13799/) (2020)
- [Tabular Benchmarks for Joint Architecture and Hyperparameter Optimization](https://mlsanity.com/paper/1905.04970/) (2019)
- [AMLB: an AutoML Benchmark](https://mlsanity.com/paper/2207.12560/) (2022)
- [Dataset2Vec: Learning Dataset Meta-Features](https://mlsanity.com/paper/1905.11063/) (2019)
- [LightAutoML: AutoML Solution for a Large Financial Services Ecosystem](https://mlsanity.com/paper/2109.01528/) (2021)
- [Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How](https://mlsanity.com/paper/2306.03828/) (2023)
- [Practical and sample efficient zero-shot HPO](https://mlsanity.com/paper/2007.13382/) (2020)
- [Assembled-OpenML: Creating Efficient Benchmarks for Ensembles in AutoML with OpenML](https://mlsanity.com/paper/2307.00285/) (2023)
- [Naive Automated Machine Learning](https://mlsanity.com/paper/2111.14514/) (2021)
- [Multi-Objective Model Selection for Time Series Forecasting](https://mlsanity.com/paper/2202.08485/) (2022)
- [CMA-ES for Post Hoc Ensembling in AutoML: A Great Success and Salvageable Failure](https://mlsanity.com/paper/2307.00286/) (2023)

## Built on by (most cited first, 30 of 33)

- [Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models](https://mlsanity.com/paper/2510.21204/) (2025)
- [MLZero: A Multi-Agent System for End-to-end Machine Learning Automation](https://mlsanity.com/paper/2505.13941/) (2025)
- [HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Models](https://mlsanity.com/paper/2405.10299/) (2024)
- [Structured Matrix Scaling for Multi-Class Calibration](https://mlsanity.com/paper/2511.03685/) (2025)
- [From Zero to Hero: Advancing Zero-Shot Foundation Models for Tabular Outlier Detection](https://mlsanity.com/paper/2602.03018/) (2026)
- [Don't Waste Your Time: Early Stopping Cross-Validation](https://mlsanity.com/paper/2405.03389/) (2024)
- [PMLBmini: A Tabular Classification Benchmark Suite for Data-Scarce Applications](https://mlsanity.com/paper/2409.01635/) (2024)
- ["Oh LLM, I'm Asking Thee, Please Give Me a Decision Tree": Zero-Shot Decision Tree Induction and Embedding with Large Language Models](https://mlsanity.com/paper/2409.18594/) (2024)
- [Transfer Learning for Finetuning Large Language Models](https://mlsanity.com/paper/2411.01195/) (2024)
- [Overtuning in Hyperparameter Optimization](https://mlsanity.com/paper/2506.19540/) (2025)
- [MacrOData: New Benchmarks of Thousands of Datasets for Tabular Outlier Detection](https://mlsanity.com/paper/2602.09329/) (2026)
- [Put CASH on Bandits: A Max K-Armed Problem for Automated Machine Learning](https://mlsanity.com/paper/2505.05226/) (2025)
- [Ensembling Finetuned Language Models for Text Classification](https://mlsanity.com/paper/2410.19889/) (2024)
- [MultiTab: A Comprehensive Benchmark Suite for Multi-Dimensional Evaluation in Tabular Domains](https://mlsanity.com/paper/2505.14312/) (2025)
- [CalArena: A Large-Scale Post-Hoc Calibration Benchmark](https://mlsanity.com/paper/2605.30188/) (2026)
- [Squeezing Lemons with Hammers: An Evaluation of AutoML and Tabular Deep Learning for Data-Scarce Classification Applications](https://mlsanity.com/paper/2405.07662/) (2024)
- [Regularized Neural Ensemblers](https://mlsanity.com/paper/2410.04520/) (2024)
- [AutoML Benchmark with shorter time constraints and early stopping](https://mlsanity.com/paper/2504.01222/) (2025)
- [LightAutoDS-Tab: Multi-AutoML Agentic System for Tabular Data](https://mlsanity.com/paper/2507.13413/) (2025)
- [HAPEns: Hardware-Aware Post-Hoc Ensembling for Tabular Data](https://mlsanity.com/paper/2603.10582/) (2026)
- [LUCoS: Latent Unsupervised Context Selection for Tabular Foundation Models](https://mlsanity.com/paper/2605.27254/) (2026)
- [Hardware Aware Ensemble Selection for Balancing Predictive Accuracy and Cost](https://mlsanity.com/paper/2408.02280/) (2024)
- [In-Context Decision Making for Optimizing Complex AutoML Pipelines](https://mlsanity.com/paper/2508.13657/) (2025)
- [Ensemble of Pre-Trained Models for Long-Tailed Trajectory Prediction](https://mlsanity.com/paper/2509.13914/) (2025)
- [Multi-layer Stack Ensembles for Time Series Forecasting](https://mlsanity.com/paper/2511.15350/) (2025)
- [Unsupervised Ensemble Learning Through Deep Energy-based Models](https://mlsanity.com/paper/2601.20556/) (2026)
- [V-ECE: Estimating General Expected Calibration Errors](https://mlsanity.com/paper/2602.24230/) (2026)
- [An Open-Source Training Dataset for Foundation Models for Black-box Optimization](https://mlsanity.com/paper/2605.23417/) (2026)
- [BBOmix: A Tabular Benchmark for Hyperparameter Optimization of Unsupervised Biological Representation Learning](https://mlsanity.com/paper/2606.05139/) (2026)
- [Partition Scores Are Not System Scores: Deployment-Fidelity Gaps in Decomposed Algorithm Selection](https://mlsanity.com/paper/2609.13785/) (2026)

## Abstract (arXiv)

We introduce TabRepo, a new dataset of tabular model evaluations and predictions. TabRepo contains the predictions and metrics of 1310 models evaluated on 200 classification and regression datasets. We illustrate the benefit of our dataset in multiple ways. First, we show that it allows to perform analysis such as comparing Hyperparameter Optimization against current AutoML systems while also considering ensembling at marginal cost by using precomputed model predictions. Second, we show that our dataset can be readily leveraged to perform transfer-learning. In particular, we show that applying standard transfer-learning techniques allows to outperform current state-of-the-art tabular systems in accuracy, runtime and latency.