# Fitness Landscape Footprint: A Framework to Compare Neural Architecture Search Problems

arXiv: https://arxiv.org/abs/2111.01584 · HTML: https://mlsanity.com/paper/2111.01584/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 51 references that are in the map.
- First posted on arXiv 2 Nov 2021.

## Builds on (most cited first, 16 of 16)

- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [NAS-Bench-101: Towards Reproducible Neural Architecture Search](https://mlsanity.com/paper/1902.09635/) (2019)
- [Searching for A Robust Neural Architecture in Four GPU Hours](https://mlsanity.com/paper/1910.04465/) (2019)
- [A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions](https://mlsanity.com/paper/2006.02903/) (2020)
- [Evolving Deep Neural Networks](https://mlsanity.com/paper/1703.00548/) (2017)
- [NAS evaluation is frustratingly hard](https://mlsanity.com/paper/1912.12522/) (2019)
- [NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Search](https://mlsanity.com/paper/2001.10422/) (2020)
- [NAS-Bench-NLP: Neural Architecture Search Benchmark for Natural Language Processing](https://mlsanity.com/paper/2006.07116/) (2020)
- [Best Practices for Scientific Research on Neural Architecture Search](https://mlsanity.com/paper/1909.02453/) (2019)
- [Metaheuristic Design of Feedforward Neural Networks: A Review of Two Decades of Research](https://mlsanity.com/paper/1705.05584/) (2017)
- [Investigating Recurrent Neural Network Memory Structures using Neuro-Evolution](https://mlsanity.com/paper/1902.02390/) (2019)
- [Bayesian Neural Architecture Search using A Training-Free Performance Metric](https://mlsanity.com/paper/2001.10726/) (2020)
- [Representing fitness landscapes by valued constraints to understand the complexity of local search](https://mlsanity.com/paper/1907.01218/) (2019)
- [On the Exploitation of Neuroevolutionary Information: Analyzing the Past for a More Efficient Future](https://mlsanity.com/paper/2105.12836/) (2021)
- [Lessons from the Clustering Analysis of a Search Space: A Centroid-based Approach to Initializing NAS](https://mlsanity.com/paper/2108.09126/) (2021)

## Built on by (most cited first, 4 of 4)

- [HPO X ELA: Investigating Hyperparameter Optimization Landscapes by Means of Exploratory Landscape Analysis](https://mlsanity.com/paper/2208.00220/) (2022)
- [A Data-driven Approach to Neural Architecture Search Initialization](https://mlsanity.com/paper/2111.03524/) (2021)
- [Landscape of Neural Architecture Search across sensors: how much do they differ ?](https://mlsanity.com/paper/2201.06321/) (2022)
- [HPO: We won't get fooled again](https://mlsanity.com/paper/2208.03320/) (2022)

## Abstract (arXiv)

Neural architecture search is a promising area of research dedicated to automating the design of neural network models. This field is rapidly growing, with a surge of methodologies ranging from Bayesian optimization,neuroevoltion, to differentiable search, and applications in various contexts. However, despite all great advances, few studies have presented insights on the difficulty of the problem itself, thus the success (or fail) of these methodologies remains unexplained. In this sense, the field of optimization has developed methods that highlight key aspects to describe optimization problems. The fitness landscape analysis stands out when it comes to characterize reliably and quantitatively search algorithms. In this paper, we propose to use fitness landscape analysis to study a neural architecture search problem. Particularly, we introduce the fitness landscape footprint, an aggregation of eight (8)general-purpose metrics to synthesize the landscape of an architecture search problem. We studied two problems, the classical image classification benchmark CIFAR-10, and the Remote-Sensing problem So2Sat LCZ42. The results present a quantitative appraisal of the problems, allowing to characterize the relative difficulty and other characteristics, such as the ruggedness or the persistence, that helps to tailor a search strategy to the problem. Also, the footprint is a tool that enables the comparison of multiple problems.