# FuNNscope: Visual microscope for interactively exploring the loss landscape of fully connected neural networks

arXiv: https://arxiv.org/abs/2204.04511 · HTML: https://mlsanity.com/paper/2204.04511/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 9 Apr 2022.

## Builds on (most cited first, 14 of 14)

- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Sharp Minima Can Generalize For Deep Nets](https://mlsanity.com/paper/1703.04933/) (2017)
- [Essentially No Barriers in Neural Network Energy Landscape](https://mlsanity.com/paper/1803.00885/) (2018)
- [Identifying and attacking the saddle point problem in high-dimensional non-convex optimization](https://mlsanity.com/paper/1406.2572/) (2014)
- [The Loss Surfaces of Multilayer Networks](https://mlsanity.com/paper/1412.0233/) (2014)
- [Qualitatively characterizing neural network optimization problems](https://mlsanity.com/paper/1412.6544/) (2014)
- [The loss landscape of overparameterized neural networks](https://mlsanity.com/paper/1804.10200/) (2018)
- [The Goldilocks zone: Towards better understanding of neural network loss landscapes](https://mlsanity.com/paper/1807.02581/) (2018)
- [Direct-Manipulation Visualization of Deep Networks](https://mlsanity.com/paper/1708.03788/) (2017)
- [Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes](https://mlsanity.com/paper/2104.11044/) (2021)
- [What can linear interpolation of neural network loss landscapes tell us?](https://mlsanity.com/paper/2106.16004/) (2021)
- [Revisiting "Qualitatively Characterizing Neural Network Optimization Problems"](https://mlsanity.com/paper/2012.06898/) (2020)
- [LossPlot: A Better Way to Visualize Loss Landscapes](https://mlsanity.com/paper/2111.15133/) (2021)

## Built on by (most cited first, 2 of 2)

- [LossLens: Diagnostics for Machine Learning through Loss Landscape Visual Analytics](https://mlsanity.com/paper/2412.13321/) (2024)
- [Class Interference of Deep Neural Networks](https://mlsanity.com/paper/2211.01370/) (2022)

## Abstract (arXiv)

Despite their effective use in various fields, many aspects of neural networks are poorly understood. One important way to investigate the characteristics of neural networks is to explore the loss landscape. However, most models produce a high-dimensional non-convex landscape which is difficult to visualize. We discuss and extend existing visualization methods based on 1D- and 2D slicing with a novel method that approximates the actual loss landscape geometry by using charts with interpretable axes. Based on the assumption that observations on small neural networks can generalize to more complex systems and provide us with helpful insights, we focus on small models in the range of a few dozen weights, which enables computationally cheap experiments and the use of an interactive dashboard. We observe symmetries around the zero vector, the influence of different layers on the global landscape, the different weight sensitivities around a minimizer, and how gradient descent navigates high-loss obstacles. The user study resulted in an average SUS (System Usability Scale) score with suggestions for improvement and opened up a number of possible application scenarios, such as autoencoders and ensemble networks.