Analytical Solution of a Three-layer Network with a Matrix Exponential Activation Function
Not on the map. Why?
Abstract (arXiv)
In practice, deeper networks tend to be more powerful than shallow ones, but this has not been understood theoretically. In this paper, we find the analytical solution of a three-layer network with a matrix exponential activation function, i.e., f(X)=W3exp(W2exp(W1X)), X∈ mathbbCd× d have analytical solutions for the equations Y1=f(X1),Y2=f(X2) for X1,X2,Y1,Y2 with only invertible assumptions. Our proof shows the power of depth and the use of a non-linear activation function, since one layer network can only solve one equation,i.e.,Y=WX.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 9 of its 20 references that are in the map. First posted on arXiv 2 Jul 2024.
Builds on
9 of 20 references.
References · 9
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
- 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
- 2016Benefits of depth in neural networkscited by 248
- 2014Qualitatively characterizing neural network optimization problemscited by 186
- 2015The Power of Depth for Feedforward Neural Networkscited by 181
- 2018Small ReLU networks are powerful memorizers: a tight analysis of memorization capacitycited by 76
- 2017Is Second-order Information Helpful for Large-scale Visual Recognition?cited by 44
- 2020Intelligent Matrix Exponentiationcited by 1
Built on by
Not cited yet.