Predictions Based on Pixel Data: Insights from PDEs and Finite Differences
Not on the map. Why?
Abstract (arXiv)
As supported by abundant experimental evidence, neural networks are state-of-the-art for many approximation tasks in high-dimensional spaces. Still, there is a lack of a rigorous theoretical understanding of what they can approximate, at which cost, and at which accuracy. One network architecture of practical use, especially for approximation tasks involving images, is (residual) convolutional networks. However, due to the locality of the linear operators involved in these networks, their analysis is more complicated than that of fully connected neural networks. This paper deals with approximation of time sequences where each observation is a matrix. We show that with relatively small networks, we can represent exactly a class of numerical discretizations of PDEs based on the method of lines. We constructively derive these results by exploiting the connections between discrete convolution and finite difference operators. Our network architecture is inspired by those typically adopted in the approximation of time sequences. We support our theoretical results with numerical experiments simulating the linear advection, heat, and Fisher equations.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 38 of its 90 references that are in the map. First posted on arXiv 1 May 2023.
Builds on
38 of 90 references.
References · 38
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2013Intriguing properties of neural networkscited by 5,114
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2017Improved Training of Wasserstein GANscited by 2,223
- 2016Group Equivariant Convolutional Networkscited by 1,154
- 2019Hamiltonian Neural Networkscited by 702
- 2018Learning Dexterous In-Hand Manipulationcited by 643
- 2015Cyclical Learning Rates for Training Neural Networkscited by 627
- 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 603
- 2017Stable Architectures for Deep Neural Networkscited by 340
- 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 311
26 more
- 2015Deep multi-scale video prediction beyond mean square errorcited by 297
- 2021Modern Koopman Theory for Dynamical Systemscited by 269
- 2021Curriculum Learning: A Surveycited by 257
- 2018PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Networkcited by 208
- 2019Towards Physics-informed Deep Learning for Turbulent Flow Predictioncited by 203
- 2018Deep Neural Networks Motivated by Partial Differential Equationscited by 187
- 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 181
- 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 162
- 2019Symplectic Recurrent Neural Networkscited by 148
- 2020Incorporating Symmetry into Deep Dynamics Models for Improved Generalizationcited by 123
- 2020A Review on Deep Learning Techniques for Video Predictioncited by 114
- 2021PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equationscited by 106
- 2016Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with ℓ1 and ℓ0 Controlscited by 84
- 2021A Dynamical System Perspective for Lipschitz Neural Networkscited by 59
- 2019DL-PDE: Deep-learning based data-driven discovery of partial differential equations from discrete and noisy datacited by 38
- 2020Structure preserving deep learningcited by 30
- 2017Loss-Sensitive Generative Adversarial Networks on Lipschitz Densitiescited by 29
- 2020PDE-based Group Equivariant Convolutional Neural Networkscited by 28
- 2018Multi-Step Prediction of Dynamic Systems with Recurrent Neural Networkscited by 25
- 2020LagNetViP: A Lagrangian Neural Network for Video Predictioncited by 24
- 2020A generative adversarial network approach to (ensemble) weather predictioncited by 18
- 2020Comparing recurrent and convolutional neural networks for predicting wave propagationcited by 15
- 2021Effects of boundary conditions in fully convolutional networks for learning spatio-temporal dynamicscited by 14
- 2022Learning Hamiltonians of constrained mechanical systemscited by 14
- 2021Locally-symplectic neural networks for learning volume-preserving dynamicscited by 9
- 2021Hamiltonian latent operators for content and motion disentanglement in image sequencescited by 5
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.