# A Local Optima Network Analysis of the Feedforward Neural Architecture Space

arXiv: https://arxiv.org/abs/2206.06903 · HTML: https://mlsanity.com/paper/2206.06903/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 2 Jun 2022.

## Builds on (most cited first, 8 of 8)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask](https://mlsanity.com/paper/1905.01067/) (2019)
- [It's Hard for Neural Networks To Learn the Game of Life](https://mlsanity.com/paper/2009.01398/) (2020)
- [Explorations on high dimensional landscapes](https://mlsanity.com/paper/1412.6615/) (2014)
- [Loss Surface Modality of Feed-Forward Neural Network Architectures](https://mlsanity.com/paper/1905.10268/) (2019)

## Built on by (most cited first, 1 of 1)

- [Towards the Inferrence of Structural Similarity of Combinatorial Landscapes](https://mlsanity.com/paper/2312.02720/) (2023)

## Abstract (arXiv)

This study investigates the use of local optima network (LON) analysis, a derivative of the fitness landscape of candidate solutions, to characterise and visualise the neural architecture space. The search space of feedforward neural network architectures with up to three layers, each with up to 10 neurons, is fully enumerated by evaluating trained model performance on a selection of data sets. Extracted LONs, while heterogeneous across data sets, all exhibit simple global structures, with single global funnels in all cases but one. These results yield early indication that LONs may provide a viable paradigm by which to analyse and optimise neural architectures.