Two-hidden-layer ReLU neural networks and finite elements
Not on the map. Why?
Abstract (arXiv)
We point out that (continuous or discontinuous) piecewise linear functions on a convex polytope mesh can be represented by two-hidden-layer ReLU neural networks in a weak sense. In addition, the numbers of neurons of the two hidden layers required to weakly represent are accurately given based on the numbers of polytopes and hyperplanes involved in this mesh. The results naturally hold for constant and linear finite element functions. Such weak representation establishes a bridge between two-hidden-layer ReLU neural networks and finite element functions, and leads to a perspective for analyzing approximation capability of ReLU neural networks in Lp norm via finite element functions. Moreover, we discuss the strict representation for tensor finite element functions via the recent tensor neural networks.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 10 of its 24 references that are in the map. First posted on arXiv 9 Mar 2024.
Builds on
10 of 24 references.
References · 10
- 2016Error bounds for approximations with deep ReLU networkscited by 567
- 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 253
- 2022MIONet: Learning multiple-input operators via tensor productcited by 180
- 2013On the number of response regions of deep feed forward networks with piece-wise linear activationscited by 94
- 2021Towards Lower Bounds on the Depth of ReLU Neural Networkscited by 55
- 2019Nonlinear Approximation and (Deep) ReLU Networkscited by 45
- 2019The Barron Space and the Flow-induced Function Spaces for Neural Network Modelscited by 39
- 2022Improved Bounds on Neural Complexity for Representing Piecewise Linear Functionscited by 26
- 2023Deep Neural Networks and Finite Elements of Any Order on Arbitrary Dimensionscited by 8
- 2023Piecewise Linear Functions Representable with Infinite Width Shallow ReLU Neural Networkscited by 1
Built on by
Not cited yet.