# PED-ANOVA: Efficiently Quantifying Hyperparameter Importance in Arbitrary Subspaces

arXiv: https://arxiv.org/abs/2304.10255 · HTML: https://mlsanity.com/paper/2304.10255/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 23 references that are in the map.
- First posted on arXiv 20 Apr 2023.

## Builds on (most cited first, 10 of 10)

- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization](https://mlsanity.com/paper/2109.09831/) (2021)
- [TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentation](https://mlsanity.com/paper/2103.10158/) (2021)
- [On the State of the Art of Evaluation in Neural Language Models](https://mlsanity.com/paper/1707.05589/) (2017)
- [Bayesian Optimization in AlphaGo](https://mlsanity.com/paper/1812.06855/) (2018)
- [Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning](https://mlsanity.com/paper/1909.12552/) (2019)
- [Explaining Hyperparameter Optimization via Partial Dependence Plots](https://mlsanity.com/paper/2111.04820/) (2021)
- [DeepCAVE: An Interactive Analysis Tool for Automated Machine Learning](https://mlsanity.com/paper/2206.03493/) (2022)

## Built on by (most cited first, 15 of 15)

- [Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance](https://mlsanity.com/paper/2304.11127/) (2023)
- [AutoLoRA: Automatically Tuning Matrix Ranks in Low-Rank Adaptation Based on Meta Learning](https://mlsanity.com/paper/2403.09113/) (2024)
- [Position: A Call to Action for a Human-Centered AutoML Paradigm](https://mlsanity.com/paper/2406.03348/) (2024)
- [Hyperparameter Importance Analysis for Multi-Objective AutoML](https://mlsanity.com/paper/2405.07640/) (2024)
- [HyperSHAP: Shapley Values and Interactions for Explaining Hyperparameter Optimization](https://mlsanity.com/paper/2502.01276/) (2025)
- [SONATA: Self-adaptive Evolutionary Framework for Hardware-aware Neural Architecture Search](https://mlsanity.com/paper/2402.13204/) (2024)
- [Complexity Scaling Laws for Neural Models using Combinatorial Optimization](https://mlsanity.com/paper/2506.12932/) (2025)
- [Preference-Optimal Multi-Metric Weighting for Parallel Coordinate Plots](https://mlsanity.com/paper/2507.02905/) (2025)
- [Hybrid quantum tensor networks for aeroelastic applications](https://mlsanity.com/paper/2508.05169/) (2025)
- [Conditional PED-ANOVA: Hyperparameter Importance in Hierarchical & Dynamic Search Spaces](https://mlsanity.com/paper/2601.20800/) (2026)
- [Explaining AutoClustering: Uncovering Meta-Feature Contribution in AutoML for Clustering](https://mlsanity.com/paper/2602.18348/) (2026)
- [Mix, Don't Tune: Bilingual Pre-Training Outperforms Hyperparameter Search in Data-Constrained Settings](https://mlsanity.com/paper/2605.13225/) (2026)
- [Importance-Aware Scheduling for High-Dimensional Hyperparameter Optimization](https://mlsanity.com/paper/2606.10068/) (2026)
- [NeuroShield: A Device-Agnostic Foundation Model for EEG Authentication](https://mlsanity.com/paper/2606.20673/) (2026)
- [Design Choices That Matter: A Functional ANOVA Analysis for Remote Sensing Multi-Label Classification](https://mlsanity.com/paper/2608.04702/) (2026)

## Abstract (arXiv)

The recent rise in popularity of Hyperparameter Optimization (HPO) for deep learning has highlighted the role that good hyperparameter (HP) space design can play in training strong models. In turn, designing a good HP space is critically dependent on understanding the role of different HPs. This motivates research on HP Importance (HPI), e.g., with the popular method of functional ANOVA (f-ANOVA). However, the original f-ANOVA formulation is inapplicable to the subspaces most relevant to algorithm designers, such as those defined by top performance. To overcome this issue, we derive a novel formulation of f-ANOVA for arbitrary subspaces and propose an algorithm that uses Pearson divergence (PED) to enable a closed-form calculation of HPI. We demonstrate that this new algorithm, dubbed PED-ANOVA, is able to successfully identify important HPs in different subspaces while also being extremely computationally efficient.