# Hybrid Algorithm Selection and Hyperparameter Tuning on Distributed Machine Learning Resources: A Hierarchical Agent-based Approach

arXiv: https://arxiv.org/abs/2309.06604 · HTML: https://mlsanity.com/paper/2309.06604/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 86 references that are in the map.
- First posted on arXiv 12 Sep 2023.

## Builds on (most cited first, 16 of 16)

- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [Gradient-based Hyperparameter Optimization through Reversible Learning](https://mlsanity.com/paper/1502.03492/) (2015)
- [SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization](https://mlsanity.com/paper/2109.09831/) (2021)
- [Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning](https://mlsanity.com/paper/1811.12808/) (2018)
- [Provably Efficient Online Hyperparameter Optimization with Population-Based Bandits](https://mlsanity.com/paper/2002.02518/) (2020)
- [Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing](https://mlsanity.com/paper/2106.04502/) (2021)
- [Automated Dynamic Algorithm Configuration](https://mlsanity.com/paper/2205.13881/) (2022)
- [Multi-agent Dynamic Algorithm Configuration](https://mlsanity.com/paper/2210.06835/) (2022)
- [Autotune: A Derivative-free Optimization Framework for Hyperparameter Tuning](https://mlsanity.com/paper/1804.07824/) (2018)
- [Progressive Sampling-Based Bayesian Optimization for Efficient and Automatic Machine Learning Model Selection](https://mlsanity.com/paper/1812.02855/) (2018)
- [HAMLET: A Hierarchical Agent-based Machine Learning Platform](https://mlsanity.com/paper/2010.04894/) (2020)
- [Agent-based Collaborative Random Search for Hyper-parameter Tuning and Global Function Optimization](https://mlsanity.com/paper/2303.03394/) (2023)
- [Automatic Configuration of Deep Neural Networks with EGO](https://mlsanity.com/paper/1810.05526/) (2018)
- [Hierarchical Collaborative Hyper-parameter Tuning](https://mlsanity.com/paper/2205.05272/) (2022)
- [High Speed Cognitive Domain Ontologies for Asset Allocation Using Loihi Spiking Neurons](https://mlsanity.com/paper/1906.12338/) (2019)
- [HPN: Personalized Federated Hyperparameter Optimization](https://mlsanity.com/paper/2304.05195/) (2023)

## Abstract (arXiv)

Algorithm selection and hyperparameter tuning are critical steps in both academic and applied machine learning. On the other hand, these steps are becoming ever increasingly delicate due to the extensive rise in the number, diversity, and distributedness of machine learning resources. Multi-agent systems, when applied to the design of machine learning platforms, bring about several distinctive characteristics such as scalability, flexibility, and robustness, just to name a few. This paper proposes a fully automatic and collaborative agent-based mechanism for selecting distributedly organized machine learning algorithms and simultaneously tuning their hyperparameters. Our method builds upon an existing agent-based hierarchical machine-learning platform and augments its query structure to support the aforementioned functionalities without being limited to specific learning, selection, and tuning mechanisms. We have conducted theoretical assessments, formal verification, and analytical study to demonstrate the correctness, resource utilization, and computational efficiency of our technique. According to the results, our solution is totally correct and exhibits linear time and space complexity in relation to the size of available resources. To provide concrete examples of how the proposed methodologies can effectively adapt and perform across a range of algorithmic options and datasets, we have also conducted a series of experiments using a system comprised of 24 algorithms and 9 datasets.