# Reducing The Search Space For Hyperparameter Optimization Using Group Sparsity

arXiv: https://arxiv.org/abs/1904.11095 · HTML: https://mlsanity.com/paper/1904.11095/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2019.

## Built on by (most cited first, 2 of 2)

- [Predicting the utility of search spaces for black-box optimization: a simple, budget-aware approach](https://mlsanity.com/paper/2112.08250/) (2021)
- [Discovering Sparse Recovery Algorithms Using Neural Architecture Search](https://mlsanity.com/paper/2512.21563/) (2025)

## Abstract (arXiv)

We propose a new algorithm for hyperparameter selection in machine learning algorithms. The algorithm is a novel modification of Harmonica, a spectral hyperparameter selection approach using sparse recovery methods. In particular, we show that a special encoding of hyperparameter space enables a natural group-sparse recovery formulation, which when coupled with HyperBand (a multi-armed bandit strategy) leads to improvement over existing hyperparameter optimization methods such as Successive Halving and Random Search. Experimental results on image datasets such as CIFAR-10 confirm the benefits of our approach.