# Efficient Automatic CASH via Rising Bandits

arXiv: https://arxiv.org/abs/2012.04371 · HTML: https://mlsanity.com/paper/2012.04371/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2020.

## Built on by (most cited first, 30 of 31)

- [OpenBox: A Generalized Black-box Optimization Service](https://mlsanity.com/paper/2106.00421/) (2021)
- [Transfer Learning for Bayesian Optimization: A Survey](https://mlsanity.com/paper/2302.05927/) (2023)
- [Which LLM to Play? Convergence-Aware Online Model Selection with Time-Increasing Bandits](https://mlsanity.com/paper/2403.07213/) (2024)
- [VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition](https://mlsanity.com/paper/2107.08861/) (2021)
- [Stochastic Rising Bandits](https://mlsanity.com/paper/2212.03798/) (2022)
- [Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale](https://mlsanity.com/paper/2201.06834/) (2022)
- [Transfer Learning based Search Space Design for Hyperparameter Tuning](https://mlsanity.com/paper/2206.02511/) (2022)
- [OpenBox: A Python Toolkit for Generalized Black-box Optimization](https://mlsanity.com/paper/2304.13339/) (2023)
- [Never Start from Scratch: Expediting On-Device LLM Personalization via Explainable Model Selection](https://mlsanity.com/paper/2504.13938/) (2025)
- [TransBO: Hyperparameter Optimization via Two-Phase Transfer Learning](https://mlsanity.com/paper/2206.02663/) (2022)
- [DivBO: Diversity-aware CASH for Ensemble Learning](https://mlsanity.com/paper/2302.03255/) (2023)
- [Best Arm Identification for Stochastic Rising Bandits](https://mlsanity.com/paper/2302.07510/) (2023)
- [Linear Bandits with Memory: from Rotting to Rising](https://mlsanity.com/paper/2302.08345/) (2023)
- [Nearly-tight Approximation Guarantees for the Improving Multi-Armed Bandits Problem](https://mlsanity.com/paper/2404.01198/) (2024)
- [A Unified and Efficient Coordinating Framework for Autonomous DBMS Tuning](https://mlsanity.com/paper/2303.05710/) (2023)
- [Put CASH on Bandits: A Max K-Armed Problem for Automated Machine Learning](https://mlsanity.com/paper/2505.05226/) (2025)
- [Learning accurate and interpretable tree-based models](https://mlsanity.com/paper/2405.15911/) (2024)
- [LLMs as In-Context Meta-Learners for Model and Hyperparameter Selection](https://mlsanity.com/paper/2510.26510/) (2025)
- [VOCALExplore: Pay-as-You-Go Video Data Exploration and Model Building [Technical Report]](https://mlsanity.com/paper/2303.04068/) (2023)
- [Tree-Structured Synergy of Large Language Models and Bayesian Optimization for Efficient CASH](https://mlsanity.com/paper/2601.12355/) (2026)
- [Search-based Methods for Multi-Cloud Configuration](https://mlsanity.com/paper/2204.09437/) (2022)
- [Efficient End-to-End AutoML via Scalable Search Space Decomposition](https://mlsanity.com/paper/2206.09423/) (2022)
- [State-Separated SARSA: A Practical Sequential Decision-Making Algorithm with Recovering Rewards](https://mlsanity.com/paper/2403.11520/) (2024)
- [Bayesian Optimization for Simultaneous Selection of Machine Learning Algorithms and Hyperparameters on Shared Latent Space](https://mlsanity.com/paper/2502.09329/) (2025)
- [On Mitigating Affinity Bias through Bandits with Evolving Biased Feedback](https://mlsanity.com/paper/2503.05662/) (2025)
- [In-Context Decision Making for Optimizing Complex AutoML Pipelines](https://mlsanity.com/paper/2508.13657/) (2025)
- [A Review on Single-Problem Multi-Attempt Heuristic Optimization](https://mlsanity.com/paper/2509.26321/) (2025)
- [CoFEH: LLM-driven Feature Engineering Empowered by Collaborative Bayesian Hyperparameter Optimization](https://mlsanity.com/paper/2602.09851/) (2026)
- [Rising Multi-Armed Bandits with Known Horizons](https://mlsanity.com/paper/2602.10727/) (2026)
- [Trajectory-Aware Best-Arm Identification for Local Search Allocation in Bayesian Optimization and Beyond](https://mlsanity.com/paper/2605.31050/) (2026)

## Abstract (arXiv)

The Combined Algorithm Selection and Hyperparameter optimization (CASH) is one of the most fundamental problems in Automatic Machine Learning (AutoML). The existing Bayesian optimization (BO) based solutions turn the CASH problem into a Hyperparameter Optimization (HPO) problem by combining the hyperparameters of all machine learning (ML) algorithms, and use BO methods to solve it. As a result, these methods suffer from the low-efficiency problem due to the huge hyperparameter space in CASH. To alleviate this issue, we propose the alternating optimization framework, where the HPO problem for each ML algorithm and the algorithm selection problem are optimized alternately. In this framework, the BO methods are used to solve the HPO problem for each ML algorithm separately, incorporating a much smaller hyperparameter space for BO methods. Furthermore, we introduce Rising Bandits, a CASH-oriented Multi-Armed Bandits (MAB) variant, to model the algorithm selection in CASH. This framework can take the advantages of both BO in solving the HPO problem with a relatively small hyperparameter space and the MABs in accelerating the algorithm selection. Moreover, we further develop an efficient online algorithm to solve the Rising Bandits with provably theoretical guarantees. The extensive experiments on 30 OpenML datasets demonstrate the superiority of the proposed approach over the competitive baselines.