Understanding Deep Learning via Entropy Space Theory
Not on the map. Why?
Abstract (arXiv)
Deep learning is often criticized for its theoretical research lagging behind practice. To make deep learning easier to understand, the entropy space theory is first introduced here. The entropy space can cover all the possibilities of any deep learning model by topological structure. It is independent of network parameters. Through the designed fundamental operations and norm, entropy space is proven to be a normed space within the formal axiomatic framework. Based on the theory, a unified coordinate system is proposed. It can coordinatize every state of a model and rank them by compression of the maximal value of information entropy. The theory offers a novel priori framework for mathematical fundamentals of deep learning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 31 references that are in the map. First posted on arXiv 29 Aug 2026.
Builds on
12 of 31 references.
References · 12
- 2023A Survey of Large Language Modelscited by 2,576
- 2016Geometric deep learning: going beyond Euclidean datacited by 1,048
- 2017Opening the Black Box of Deep Neural Networks via Informationcited by 503
- 2021Deep learning: a statistical viewpointcited by 240
- 2020When Do Neural Networks Outperform Kernel Methods?cited by 150
- 2021Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolationcited by 132
- 2022Analyzing Transformers in Embedding Spacecited by 132
- 2020Recent advances in deep learning theorycited by 41
- 2019A Selective Overview of Deep Learningcited by 31
- 2019On Reproducing Kernel Banach Spaces: Generic Definitions and Unified Framework of Constructionscited by 29
- 2024A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Modelscited by 20
- 2025Parameter Symmetry Potentially Unifies Deep Learning Theorycited by 12
Built on by
Not cited yet.