mlsanityData to 7 Oct 2026

Neuro-Visualizer: An Auto-encoder-based Loss Landscape Visualization Method

arXiv 2309.14601 ↗First posted 26 Sep 2023cs.LG cs.HCcited by 0

Not on the map. Why?

Abstract (arXiv)

In recent years, there has been a growing interest in visualizing the loss landscape of neural networks. Linear landscape visualization methods, such as principal component analysis, have become widely used as they intuitively help researchers study neural networks and their training process. However, these linear methods suffer from limitations and drawbacks due to their lack of flexibility and low fidelity at representing the high dimensional landscape. In this paper, we present a novel auto-encoder-based non-linear landscape visualization method called Neuro-Visualizer that addresses these shortcoming and provides useful insights about neural network loss landscapes. To demonstrate its potential, we run experiments on a variety of problems in two separate applications of knowledge-guided machine learning (KGML). Our findings show that Neuro-Visualizer outperforms other linear and non-linear baselines and helps corroborate, and sometime challenge, claims proposed by machine learning community. All code and data used in the experiments of this paper are available at an anonymous link https://anonymous.4open.science/r/NeuroVisualizer-FDD6

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 43 references that are in the map. First posted on arXiv 26 Sep 2023.

Builds on

29 of 43 references.

References · 29

  1. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  2. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  3. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  4. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  5. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  6. 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
  7. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  8. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  9. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 463
  10. 2017Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equationscited by 362
  11. 2019PyHessian: Neural Networks Through the Lens of the Hessiancited by 291
  12. 2014Qualitatively characterizing neural network optimization problemscited by 186
17 more
  1. 2020PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domaincited by 173
  2. 2016Theory-guided Data Science: A New Paradigm for Scientific Discovery from Datacited by 169
  3. 2017Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equationscited by 158
  4. 2021Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learningcited by 100
  5. 2020The Global Landscape of Neural Networks: An Overviewcited by 57
  6. 2019Understanding Generalization through Visualizationscited by 54
  7. 2022Beyond the Quadratic Approximation: the Multiscale Structure of Neural Network Loss Landscapescited by 42
  8. 2018On the loss landscape of a class of deep neural networks with no bad local valleyscited by 38
  9. 2021Taxonomizing local versus global structure in neural network loss landscapescited by 34
  10. 2018SmoothOut: Smoothing Out Sharp Minima to Improve Generalization in Deep Learningcited by 25
  11. 2019The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networkscited by 20
  12. 2021Exploring the Geometry and Topology of Neural Network Loss Landscapescited by 14
  13. 2019Towards Understanding Generalization in Gradient-Based Meta-Learningcited by 12
  14. 2019Understanding Adversarial Robustness Through Loss Landscape Geometriescited by 12
  15. 2020CoPhy-PGNN: Learning Physics-guided Neural Networks with Competing Loss Functions for Solving Eigenvalue Problemscited by 11
  16. 2020A Simple General Approach to Balance Task Difficulty in Multi-Task Learningcited by 8
  17. 2019GradVis: Visualization and Second Order Analysis of Optimization Surfaces during the Training of Deep Neural Networkscited by 5

Built on by

Not cited yet.