# SPINN: Sparse, Physics-based, and partially Interpretable Neural Networks for PDEs

arXiv: https://arxiv.org/abs/2102.13037 · HTML: https://mlsanity.com/paper/2102.13037/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 44 references that are in the map.
- First posted on arXiv 25 Feb 2021.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [A unified deep artificial neural network approach to partial differential equations in complex geometries](https://mlsanity.com/paper/1711.06464/) (2017)
- [An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications](https://mlsanity.com/paper/1908.10407/) (2019)
- [Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems](https://mlsanity.com/paper/1801.01236/) (2018)
- [Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains](https://mlsanity.com/paper/2007.11207/) (2020)
- [Physics Informed Extreme Learning Machine (PIELM) -- A rapid method for the numerical solution of partial differential equations](https://mlsanity.com/paper/1907.03507/) (2019)
- [Machine Learning from a Continuous Viewpoint](https://mlsanity.com/paper/1912.12777/) (2019)
- [Robust Training and Initialization of Deep Neural Networks: An Adaptive Basis Viewpoint](https://mlsanity.com/paper/1912.04862/) (2019)
- [Deep least-squares methods: an unsupervised learning-based numerical method for solving elliptic PDEs](https://mlsanity.com/paper/1911.02109/) (2019)
- [Partition of unity networks: deep hp-approximation](https://mlsanity.com/paper/2101.11256/) (2021)
- [Growing axons: greedy learning of neural networks with application to function approximation](https://mlsanity.com/paper/1910.12686/) (2019)
- [automan: a simple, Python-based, automation framework for numerical computing](https://mlsanity.com/paper/1712.04786/) (2017)

## Built on by (most cited first, 30 of 32)

- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Physics and Equality Constrained Artificial Neural Networks: Application to Forward and Inverse Problems with Multi-fidelity Data Fusion](https://mlsanity.com/paper/2109.14860/) (2021)
- [Adaptive Self-supervision Algorithms for Physics-informed Neural Networks](https://mlsanity.com/paper/2207.04084/) (2022)
- [Neural (Tangent Kernel) Collapse](https://mlsanity.com/paper/2305.16427/) (2023)
- [Learning Stochastic Dynamics with Statistics-Informed Neural Network](https://mlsanity.com/paper/2202.12278/) (2022)
- [RANG: A Residual-based Adaptive Node Generation Method for Physics-Informed Neural Networks](https://mlsanity.com/paper/2205.01051/) (2022)
- [NeuFENet: Neural Finite Element Solutions with Theoretical Bounds for Parametric PDEs](https://mlsanity.com/paper/2110.01601/) (2021)
- [Multilevel CNNs for Parametric PDEs](https://mlsanity.com/paper/2304.00388/) (2023)
- [Binary structured physics-informed neural networks for solving equations with rapidly changing solutions](https://mlsanity.com/paper/2401.12806/) (2024)
- [Med-Real2Sim: Non-Invasive Medical Digital Twins using Physics-Informed Self-Supervised Learning](https://mlsanity.com/paper/2403.00177/) (2024)
- [Curriculum Learning-Driven PIELMs for Fluid Flow Simulations](https://mlsanity.com/paper/2503.06347/) (2025)
- [Neural PDE Solvers for Irregular Domains](https://mlsanity.com/paper/2211.03241/) (2022)
- [Neural Control of Parametric Solutions for High-dimensional Evolution PDEs](https://mlsanity.com/paper/2302.00045/) (2023)
- [Approximating High-Dimensional Minimal Surfaces with Physics-Informed Neural Networks](https://mlsanity.com/paper/2309.02589/) (2023)
- [Learning nonlinear integral operators via Recurrent Neural Networks and its application in solving Integro-Differential Equations](https://mlsanity.com/paper/2310.09434/) (2023)
- [Learning Domain-Independent Green's Function For Elliptic Partial Differential Equations](https://mlsanity.com/paper/2401.17172/) (2024)
- [Robust Regression with Highly Corrupted Data via Physics Informed Neural Networks](https://mlsanity.com/paper/2210.10646/) (2022)
- [LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on Complex Geometry](https://mlsanity.com/paper/2302.01518/) (2023)
- [AB-PINNs: Adaptive-Basis Physics-Informed Neural Networks for Residual-Driven Domain Decomposition](https://mlsanity.com/paper/2510.08924/) (2025)
- [Free-RBF-KAN: Kolmogorov-Arnold Networks with Adaptive Radial Basis Functions for Efficient Function Learning](https://mlsanity.com/paper/2601.07760/) (2026)
- [Constrained or Unconstrained? Neural-Network-Based Equation Discovery from Data](https://mlsanity.com/paper/2406.02581/) (2024)
- [Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm](https://mlsanity.com/paper/2602.08515/) (2026)
- [Surrogates for Physics-based and Data-driven Modelling of Parametric Systems: Review and New Perspectives](https://mlsanity.com/paper/2603.12870/) (2026)
- [Harnessing physics-informed operators for high-dimensional reliability analysis problems](https://mlsanity.com/paper/2409.04708/) (2024)
- [PINNfluence: Interpreting PINNs through Influence Functions](https://mlsanity.com/paper/2409.08958/) (2024)
- [FinNet: Solving Time-Independent Differential Equations with Finite Difference Neural Network](https://mlsanity.com/paper/2202.09282/) (2022)
- [Anisotropic, Sparse and Interpretable Physics-Informed Neural Networks for PDEs](https://mlsanity.com/paper/2207.00377/) (2022)
- [Solving Coupled Differential Equation Groups Using PINO-CDE](https://mlsanity.com/paper/2210.00222/) (2022)
- [Approximation of Solution Operators for High-dimensional PDEs](https://mlsanity.com/paper/2401.10385/) (2024)
- [A Physics-driven GraphSAGE Method for Physical Process Simulations Described by Partial Differential Equations](https://mlsanity.com/paper/2403.08569/) (2024)

## Abstract (arXiv)

We introduce a class of Sparse, Physics-based, and partially Interpretable Neural Networks (SPINN) for solving ordinary and partial differential equations (PDEs). By reinterpreting a traditional meshless representation of solutions of PDEs we develop a class of sparse neural network architectures that are partially interpretable. The SPINN model we propose here serves as a seamless bridge between two extreme modeling tools for PDEs, namely dense neural network based methods like Physics Informed Neural Networks (PINNs) and traditional mesh-free numerical methods, thereby providing a novel means to develop a new class of hybrid algorithms that build on the best of both these viewpoints. A unique feature of the SPINN model that distinguishes it from other neural network based approximations proposed earlier is that it is (i) interpretable, in a particular sense made precise in the work, and (ii) sparse in the sense that it has much fewer connections than typical dense neural networks used for PDEs. Further, the SPINN algorithm implicitly encodes mesh adaptivity and is able to handle discontinuities in the solutions. In addition, we demonstrate that Fourier series representations can also be expressed as a special class of SPINN and propose generalized neural network analogues of Fourier representations. We illustrate the utility of the proposed method with a variety of examples involving ordinary differential equations, elliptic, parabolic, hyperbolic and nonlinear partial differential equations, and an example in fluid dynamics.