# Least-Squares ReLU Neural Network (LSNN) Method For Linear Advection-Reaction Equation

arXiv: https://arxiv.org/abs/2105.11632 · HTML: https://mlsanity.com/paper/2105.11632/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 30 references that are in the map.
- First posted on arXiv 25 May 2021.

## Builds on (most cited first, 4 of 4)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Deep Network Approximation Characterized by Number of Neurons](https://mlsanity.com/paper/1906.05497/) (2019)
- [Deep least-squares methods: an unsupervised learning-based numerical method for solving elliptic PDEs](https://mlsanity.com/paper/1911.02109/) (2019)

## Built on by (most cited first, 16 of 16)

- [Bounding the Rademacher Complexity of Fourier neural operators](https://mlsanity.com/paper/2209.05150/) (2022)
- [Approximating Numerical Fluxes Using Fourier Neural Operators for Hyperbolic Conservation Laws](https://mlsanity.com/paper/2401.01783/) (2024)
- [DeepParticle: learning invariant measure by a deep neural network minimizing Wasserstein distance on data generated from an interacting particle method](https://mlsanity.com/paper/2111.01356/) (2021)
- [Solving Elliptic Problems with Singular Sources using Singularity Splitting Deep Ritz Method](https://mlsanity.com/paper/2209.02931/) (2022)
- [Capturing the Diffusive Behavior of the Multiscale Linear Transport Equations by Asymptotic-Preserving Convolutional DeepONets](https://mlsanity.com/paper/2306.15891/) (2023)
- [Solving Elliptic Optimal Control Problems via Neural Networks and Optimality System](https://mlsanity.com/paper/2308.11925/) (2023)
- [A Structure-Guided Gauss-Newton Method for Shallow ReLU Neural Network](https://mlsanity.com/paper/2404.05064/) (2024)
- [Self-adaptive deep neural network: Numerical approximation to functions and PDEs](https://mlsanity.com/paper/2109.02839/) (2021)
- [Least-Squares Neural Network (LSNN) Method For Scalar Nonlinear Hyperbolic Conservation Laws: Discrete Divergence Operator](https://mlsanity.com/paper/2110.10895/) (2021)
- [A variational neural network approach for glacier modelling with nonlinear rheology](https://mlsanity.com/paper/2209.02088/) (2022)
- [Cell-average based neural network method for hyperbolic and parabolic partial differential equations](https://mlsanity.com/paper/2107.00813/) (2021)
- [Neural Networks Trained by Weight Permutation are Universal Approximators](https://mlsanity.com/paper/2407.01033/) (2024)
- [An Iterative Deep Ritz Method for Monotone Elliptic Problems](https://mlsanity.com/paper/2501.15186/) (2025)
- [Macroscopic auxiliary asymptotic preserving neural networks for the linear radiative transfer equations](https://mlsanity.com/paper/2403.01820/) (2024)
- [Efficient Shallow Ritz Method For 1D Diffusion-Reaction Problems](https://mlsanity.com/paper/2407.01496/) (2024)
- [ReLU neural network approximation to piecewise constant functions](https://mlsanity.com/paper/2410.16506/) (2024)

## Abstract (arXiv)

This paper studies least-squares ReLU neural network method for solving the linear advection-reaction problem with discontinuous solution. The method is a discretization of an equivalent least-squares formulation in the set of neural network functions with the ReLU activation function. The method is capable of approximating the discontinuous interface of the underlying problem automatically through the free hyper-planes of the ReLU neural network and, hence, outperforms mesh-based numerical methods in terms of the number of degrees of freedom. Numerical results of some benchmark test problems show that the method can not only approximate the solution with the least number of parameters, but also avoid the common Gibbs phenomena along the discontinuous interface. Moreover, a three-layer ReLU neural network is necessary and sufficient in order to well approximate a discontinuous solution with an interface in ℝ2 that is not a straight line.