# Hyperparameter Search in Machine Learning

arXiv: https://arxiv.org/abs/1502.02127 · HTML: https://mlsanity.com/paper/1502.02127/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Feb 2015.

## Built on by (most cited first, 30 of 33)

- [The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanations](https://mlsanity.com/paper/2106.00786/) (2021)
- [Super Consistency of Neural Network Landscapes and Learning Rate Transfer](https://mlsanity.com/paper/2402.17457/) (2024)
- [Hyperparameter Optimization Is Deceiving Us, and How to Stop It](https://mlsanity.com/paper/2102.03034/) (2021)
- [Edge-Native Intelligence for 6G Communications Driven by Federated Learning: A Survey of Trends and Challenges](https://mlsanity.com/paper/2111.07392/) (2021)
- [An Overview of Machine Learning Techniques for Radiowave Propagation Modeling](https://mlsanity.com/paper/2101.11760/) (2021)
- [VisRuler: Visual Analytics for Extracting Decision Rules from Bagged and Boosted Decision Trees](https://mlsanity.com/paper/2112.00334/) (2021)
- [A Novel Genetic Algorithm with Hierarchical Evaluation Strategy for Hyperparameter Optimisation of Graph Neural Networks](https://mlsanity.com/paper/2101.09300/) (2021)
- [KGTuner: Efficient Hyper-parameter Search for Knowledge Graph Learning](https://mlsanity.com/paper/2205.02460/) (2022)
- [Distributed intelligence on the Edge-to-Cloud Continuum: A systematic literature review](https://mlsanity.com/paper/2205.01081/) (2022)
- [An Efficient Drifters Deployment Strategy to Evaluate Water Current Velocity Fields](https://mlsanity.com/paper/2301.04216/) (2023)
- [AutoML for Large Capacity Modeling of Meta's Ranking Systems](https://mlsanity.com/paper/2311.07870/) (2023)
- [Machine Learning Approaches for Mental Illness Detection on Social Media: A Systematic Review of Biases and Methodological Challenges](https://mlsanity.com/paper/2410.16204/) (2024)
- [Efficient and Joint Hyperparameter and Architecture Search for Collaborative Filtering](https://mlsanity.com/paper/2307.11004/) (2023)
- [Characterizing Social Imaginaries and Self-Disclosures of Dissonance in Online Conspiracy Discussion Communities](https://mlsanity.com/paper/2107.10204/) (2021)
- [Jasmine: A New Active Learning Approach to Combat Cybercrime](https://mlsanity.com/paper/2108.06238/) (2021)
- [HyP-ABC: A Novel Automated Hyper-Parameter Tuning Algorithm Using Evolutionary Optimization](https://mlsanity.com/paper/2109.05319/) (2021)
- [Nothing makes sense in deep learning, except in the light of evolution](https://mlsanity.com/paper/2205.10320/) (2022)
- [Boosting House Price Estimations with Multi-Head Gated Attention](https://mlsanity.com/paper/2405.07456/) (2024)
- [Hierarchical Deep Feature Fusion and Ensemble Learning for Enhanced Brain Tumor MRI Classification](https://mlsanity.com/paper/2506.12363/) (2025)
- [Practitioner Motives to Use Different Hyperparameter Optimization Methods](https://mlsanity.com/paper/2203.01717/) (2022)
- [Forecasting the abnormal events at well drilling with machine learning](https://mlsanity.com/paper/2203.05378/) (2022)
- [Mithridates: Auditing and Boosting Backdoor Resistance of Machine Learning Pipelines](https://mlsanity.com/paper/2302.04977/) (2023)
- [GenAI4UQ: A Software for Inverse Uncertainty Quantification Using Conditional Generative Models](https://mlsanity.com/paper/2412.07026/) (2024)
- [A Unified Hyperparameter Optimization Pipeline for Transformer-Based Time Series Forecasting Models](https://mlsanity.com/paper/2501.01394/) (2025)
- [Hybrid Ensemble Approaches: Optimal Deep Feature Fusion and Hyperparameter-Tuned Classifier Ensembling for Enhanced Brain Tumor Classification](https://mlsanity.com/paper/2507.12177/) (2025)
- [Artificial Intelligence and Design of Experiments for Assessing Security of Electricity Supply: A Review and Strategic Outlook](https://mlsanity.com/paper/2112.04889/) (2021)
- [Density reconstruction from schlieren images through Bayesian nonparametric models](https://mlsanity.com/paper/2201.05233/) (2022)
- [FairPilot: An Explorative System for Hyperparameter Tuning through the Lens of Fairness](https://mlsanity.com/paper/2304.04679/) (2023)
- [Evolution under Length Constraints for CNN Architecture design](https://mlsanity.com/paper/2305.03416/) (2023)
- [Towards Sobolev Pruning](https://mlsanity.com/paper/2312.03510/) (2023)

## Abstract (arXiv)

We introduce the hyperparameter search problem in the field of machine learning and discuss its main challenges from an optimization perspective. Machine learning methods attempt to build models that capture some element of interest based on given data. Most common learning algorithms feature a set of hyperparameters that must be determined before training commences. The choice of hyperparameters can significantly affect the resulting model's performance, but determining good values can be complex; hence a disciplined, theoretically sound search strategy is essential.