# Neural Networks for Singular Perturbations

arXiv: https://arxiv.org/abs/2401.06656 · HTML: https://mlsanity.com/paper/2401.06656/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 27 references that are in the map.
- First posted on arXiv 12 Jan 2024.

## Builds on (most cited first, 7 of 7)

- [Optimal approximation of piecewise smooth functions using deep ReLU neural networks](https://mlsanity.com/paper/1709.05289/) (2017)
- [On the approximation of functions by tanh neural networks](https://mlsanity.com/paper/2104.08938/) (2021)
- [Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations](https://mlsanity.com/paper/2112.08125/) (2021)
- [Exponential ReLU Neural Network Approximation Rates for Point and Edge Singularities](https://mlsanity.com/paper/2010.12217/) (2020)
- [An Exact Mapping From ReLU Networks to Spiking Neural Networks](https://mlsanity.com/paper/2212.12522/) (2022)
- [Deep ReLU networks and high-order finite element methods II: Chebyshev emulation](https://mlsanity.com/paper/2310.07261/) (2023)
- [Rethinking Skip Connections in Spiking Neural Networks with Time-To-First-Spike Coding](https://mlsanity.com/paper/2312.00919/) (2023)

## Built on by (most cited first, 1 of 1)

- [Exponential Expressivity of ReLUk Neural Networks on Gevrey Classes with Point Singularities](https://mlsanity.com/paper/2403.02035/) (2024)

## Abstract (arXiv)

We prove deep neural network (DNN for short) expressivity rate bounds for solution sets of a model class of singularly perturbed, elliptic two-point boundary value problems, in Sobolev norms, on the bounded interval (-1,1). We assume that the given source term and reaction coefficient are analytic in [-1,1]. We establish expression rate bounds in Sobolev norms in terms of the NN size which are uniform with respect to the singular perturbation parameter for several classes of DNN architectures. In particular, ReLU NNs, spiking NNs, and tanh- and sigmoid-activated NNs. The latter activations can represent ``exponential boundary layer solution features'' explicitly, in the last hidden layer of the DNN, i.e. in a shallow subnetwork, and afford improved robust expression rate bounds in terms of the NN size. We prove that all DNN architectures allow robust exponential solution expression in so-called `energy' as well as in `balanced' Sobolev norms, for analytic input data.