Exponential Expressivity of ReLUk Neural Networks on Gevrey Classes with Point Singularities

We analyze deep Neural Network emulation rates of smooth functions with point singularities in bounded, polytopal domains D subset ℝd, d=2,3. We prove exponential emulation rates in Sobolev spaces in terms of the number of neurons and in terms of the number of nonzero coefficients for Gevrey-regular solution classes defined in terms of weighted Sobolev scales in D, comprising the countably-normed spaces of I.M. Babuvska and B.Q. Guo. As intermediate result, we prove that continuous, piecewise polynomial high order (``p-version'') finite elements with elementwise polynomial degree p∈mathbbN on arbitrary, regular, simplicial partitions of polyhedral domains D subset ℝd, d≥ 2 can be exactly emulated by neural networks combining ReLU and ReLU2 activations. On shape-regular, simplicial partitions of polytopal domains D, both the number of neurons and the number of nonzero parameters are proportional to the number of degrees of freedom of the finite element space, in particular for the hp-Finite Element Method of I.M. Babuvska and B.Q. Guo.