mlsanityData to 7 Oct 2026

Consolidated learning -- a domain-specific model-free optimization strategy with examples for XGBoost and MIMIC-IV

arXiv 2201.11815 ↗First posted 27 Jan 2022cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

For many machine learning models, a choice of hyperparameters is a crucial step towards achieving high performance. Prevalent meta-learning approaches focus on obtaining good hyperparameters configurations with a limited computational budget for a completely new task based on the results obtained from the prior tasks. This paper proposes a new formulation of the tuning problem, called consolidated learning, more suited to practical challenges faced by model developers, in which a large number of predictive models are created on similar data sets. In such settings, we are interested in the total optimization time rather than tuning for a single task. We show that a carefully selected static portfolio of hyperparameters yields good results for anytime optimization, maintaining ease of use and implementation. Moreover, we point out how to construct such a portfolio for specific domains. The improvement in the optimization is possible due to more efficient transfer of hyperparameter configurations between similar tasks. We demonstrate the effectiveness of this approach through an empirical study for XGBoost algorithm and the collection of predictive tasks extracted from the MIMIC-IV medical database; however, consolidated learning is applicable in many others fields.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 18 of its 50 references that are in the map. First posted on arXiv 27 Jan 2022.

Builds on

18 of 50 references.

References · 18

  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  2. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
  3. 2018All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneouslycited by 364
  4. 2015Non-stochastic Best Arm Identification and Hyperparameter Optimizationcited by 219
  5. 2020Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learningcited by 174
  6. 2016Towards a Neural Statisticiancited by 119
  7. 2018Tunability: Importance of Hyperparameters of Machine Learning Algorithmscited by 98
  8. 2018BOCK : Bayesian Optimization with Cylindrical Kernelscited by 62
  9. 2019Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learningcited by 50
  10. 2019Dataset2Vec: Learning Dataset Meta-Featurescited by 37
  11. 2021LightAutoML: AutoML Solution for a Large Financial Services Ecosystemcited by 32
  12. 2018The Variational Homoencoder: Learning to learn high capacity generative models from few examplescited by 23
6 more
  1. 2018AutoPrognosis: Automated Clinical Prognostic Modeling via Bayesian Optimization with Structured Kernel Learningcited by 22
  2. 2021MIMIC-IF: Interpretability and Fairness Evaluation of Deep Learning Models on MIMIC-IV Datasetcited by 15
  3. 2021Machine learning for modeling the progression of Alzheimer disease dementia using clinical data: a systematic literature reviewcited by 13
  4. 2018Learning Multiple Defaults for Machine Learning Algorithmscited by 10
  5. 2020Towards explainable meta-learningcited by 4
  6. 2020MementoML: Performance of selected machine learning algorithm configurations on OpenML100 datasetscited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1.

2 papers.

  1. 2022Transfer Learning with Deep Tabular Modelscited by 52
  2. 2023SeFNet: Bridging Tabular Datasets with Semantic Feature Netscited by 1