mlsanityData to 7 Oct 2026

OpenML Benchmarking Suites

arXiv 1708.03731 ↗First posted 11 Aug 2017stat.ML cs.LGcited by 169

Not on the map. Why?

Abstract (arXiv)

Machine learning research depends on objectively interpretable, comparable, and reproducible algorithm benchmarks. We advocate the use of curated, comprehensive suites of machine learning tasks to standardize the setup, execution, and reporting of benchmarks. We enable this through software tools that help to create and leverage these benchmarking suites. These are seamlessly integrated into the OpenML platform, and accessible through interfaces in Python, Java, and R. OpenML benchmarking suites (a) are easy to use through standardized data formats, APIs, and client libraries; (b) come with extensive meta-information on the included datasets; and (c) allow benchmarks to be shared and reused in future studies. We then present a first, carefully curated and practical benchmarking suite for classification: the OpenML Curated Classification benchmarking suite 2018 (OpenML-CC18). Finally, we discuss use cases and applications which demonstrate the usefulness of OpenML benchmarking suites and the OpenML-CC18 in particular.

Key facts. Cited by 169 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Aug 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 16; 2023: 21; 2024: 35; 2025: 37; 2026: 50.

169 papers.

Sort:

By area:

  1. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
  2. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 218
  3. 2022Why do tree-based models still outperform deep learning on tabular data?cited by 198
  4. 2021SCARF: Self-Supervised Contrastive Learning using Random Feature Corruptioncited by 151
  5. 2021HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPOcited by 111
  6. 2021SubTab: Subsetting Features of Tabular Data for Self-Supervised Representation Learningcited by 101
  7. 2024TabDPT: Scaling Tabular Foundation Models on Real Datacited by 97
  8. 2024Large Scale Transfer Learning for Tabular Data via Language Modelingcited by 73
  9. 2024Retrieval & Fine-Tuning for In-Context Tabular Modelscited by 58
  10. 2022AMLB: an AutoML Benchmarkcited by 55
  11. 2022OpenFE: Automated Feature Generation with Expert-level Performancecited by 52
  12. 2023STUNT: Few-shot Tabular Learning with Self-generated Tasks from Unlabeled Tablescited by 48
38 more
  1. 2024Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoningcited by 48
  2. 2025CausalPFN: Amortized Causal Effect Estimation via In-Context Learningcited by 43
  3. 2024shapiq: Shapley Interactions for Machine Learningcited by 42
  4. 2021Post-Contextual-Bandit Inferencecited by 32
  5. 2025ConTextTab: A Semantics-Aware Tabular In-Context Learnercited by 31
  6. 2024HyperFast: Instant Classification for Tabular Datacited by 22
  7. 2023Sampling weights of deep neural networkscited by 20
  8. 2023Interpretable Deep Clustering for Tabular Datacited by 19
  9. 2023MotherNet: Fast Training and Inference via Hyper-Network Transformerscited by 19
  10. 2024In-Context Data Distillation with TabPFNcited by 19
  11. 2024Position: Why We Must Rethink Empirical Research in Machine Learningcited by 19
  12. 2022End-to-end Optimization of Machine Learning Prediction Queriescited by 16
  13. 2024A Survey on Self-Supervised Learning for Non-Sequential Tabular Datacited by 15
  14. 2024A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Datacited by 15
  15. 2024TabEBM: A Tabular Data Augmentation Method with Distinct Class-Specific Energy-Based Modelscited by 15
  16. 2025Orion-MSP: Multi-Scale Sparse Attention for Tabular In-Context Learningcited by 15
  17. 2025Amortized In-Context Bayesian Posterior Estimationcited by 13
  18. 2025TabFlex: Scaling Tabular Learning to Millions with Linear Attentioncited by 13
  19. 2023Deep Clustering of Tabular Data by Weighted Gaussian Distribution Learningcited by 12
  20. 2024BEExAI: Benchmark to Evaluate Explainable AIcited by 11
  21. 2021Risk Minimization from Adaptively Collected Data: Guarantees for Supervised and Policy Learningcited by 10
  22. 2023GRANDE: Gradient-Based Decision Tree Ensembles for Tabular Datacited by 10
  23. 2024Interpretable Machine Learning for TabPFNcited by 10
  24. 2024Benchmark Data Repositories for Better Benchmarkingcited by 10
  25. 2025TabTune: A Unified Library for Inference and Fine-Tuning Tabular Foundation Modelscited by 10
  26. 2022GANDALF: Gated Adaptive Network for Deep Automated Learning of Featurescited by 9
  27. 2022HPO X ELA: Investigating Hyperparameter Optimization Landscapes by Means of Exploratory Landscape Analysiscited by 9
  28. 2022A Bayesian Bradley-Terry model to compare multiple ML algorithms on multiple data setscited by 9
  29. 2023A Hyperparameter Study for Quantum Kernel Methodscited by 9
  30. 2024Attention versus Contrastive Learning of Tabular Data -- A Data-centric Benchmarkingcited by 9
  31. 2026Exploring Fine-Tuning for Tabular Foundation Modelscited by 9
  32. 2022SubStrat: A Subset-Based Strategy for Faster AutoMLcited by 8
  33. 2023Assembled-OpenML: Creating Efficient Benchmarks for Ensembles in AutoML with OpenMLcited by 8
  34. 2025Zero-shot Meta-learning for Tabular Prediction Tasks with Adversarially Pre-trained Transformercited by 8
  35. 2026From Zero to Hero: Advancing Zero-Shot Foundation Models for Tabular Outlier Detectioncited by 8
  36. 2022Evaluating histopathology transfer learning with ChampKitcited by 7
  37. 2022Interaction-Grounded Learning with Action-inclusive Feedbackcited by 7
  38. 2022Hyperparameter Importance of Quantum Neural Networks Across Small Datasetscited by 7

50 most cited of 169 All 169 (JSON)