# Structure preserving deep learning

arXiv: https://arxiv.org/abs/2006.03364 · HTML: https://mlsanity.com/paper/2006.03364/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2020.

## Built on by (most cited first, 30 of 31)

- [The mpEDMD Algorithm for Data-Driven Computations of Measure-Preserving Dynamical Systems](https://mlsanity.com/paper/2209.02244/) (2022)
- [PottsMGNet: A Mathematical Explanation of Encoder-Decoder Based Neural Networks](https://mlsanity.com/paper/2307.09039/) (2023)
- [Exactly conservative physics-informed neural networks and deep operator networks for dynamical systems](https://mlsanity.com/paper/2311.14131/) (2023)
- [Physics perception in sloshing scenes with guaranteed thermodynamic consistency](https://mlsanity.com/paper/2106.13301/) (2021)
- [Dynamical systems' based neural networks](https://mlsanity.com/paper/2210.02373/) (2022)
- [Designing Stable Neural Networks using Convex Analysis and ODEs](https://mlsanity.com/paper/2306.17332/) (2023)
- [Locally-symplectic neural networks for learning volume-preserving dynamics](https://mlsanity.com/paper/2109.09151/) (2021)
- [Mimetic Neural Networks: A unified framework for Protein Design and Folding](https://mlsanity.com/paper/2102.03881/) (2021)
- [Sparsity in long-time control of neural ODEs](https://mlsanity.com/paper/2102.13566/) (2021)
- [Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents](https://mlsanity.com/paper/2203.10592/) (2022)
- [Hierarchical Training of Deep Neural Networks Using Early Exiting](https://mlsanity.com/paper/2303.02384/) (2023)
- [Symplectic Neural Flows for Modeling and Discovery](https://mlsanity.com/paper/2412.16787/) (2024)
- [VPNets: Volume-preserving neural networks for learning source-free dynamics](https://mlsanity.com/paper/2204.13843/) (2022)
- [Deep neural networks on diffeomorphism groups for optimal shape reparameterization](https://mlsanity.com/paper/2207.11141/) (2022)
- [Predictions Based on Pixel Data: Insights from PDEs and Finite Differences](https://mlsanity.com/paper/2305.00723/) (2023)
- [Do stable neural networks exist for classification problems? -- A new view on stability in AI](https://mlsanity.com/paper/2401.07874/) (2024)
- [Approximation properties of neural ODEs](https://mlsanity.com/paper/2503.15696/) (2025)
- [A Discrete Variational Derivation of Accelerated Methods in Optimization](https://mlsanity.com/paper/2106.02700/) (2021)
- [Blessing of Dimensionality for Approximating Sobolev Classes on Manifolds](https://mlsanity.com/paper/2408.06996/) (2024)
- [Control, Optimal Transport and Neural Differential Equations in Supervised Learning](https://mlsanity.com/paper/2503.15105/) (2025)
- [Image Segmentation via Variational Model Based Tailored UNet: A Deep Variational Framework](https://mlsanity.com/paper/2505.05806/) (2025)
- [Nonlinear GENERIC-Embedded Neural Networks (N-GENNs): Learning GENERIC dynamics with non-quadratic dissipation potentials](https://mlsanity.com/paper/2605.09058/) (2026)
- [CSympNet-ID: conformal-symplectic map learning for linearly damped Hamiltonian systems](https://mlsanity.com/paper/2607.03339/) (2026)
- [Classification with Runge-Kutta networks and feature space augmentation](https://mlsanity.com/paper/2104.02369/) (2021)
- [Inverse Evolution Layers: Physics-informed Regularizers for Deep Neural Networks](https://mlsanity.com/paper/2307.07344/) (2023)
- [Improving regional weather forecasts with neural interpolation](https://mlsanity.com/paper/2505.12040/) (2025)
- [Systematic Performance Assessment of Deep Material Networks for Multiscale Material Modeling](https://mlsanity.com/paper/2602.07192/) (2026)
- [Constructive interpolation and generalization rates for neural ODEs: a control perspective](https://mlsanity.com/paper/2606.00469/) (2026)
- [Learning Forced Multibody Dynamics on Lie Groups](https://mlsanity.com/paper/2607.12627/) (2026)
- [Structure-preserving uncertainty quantification for GENERIC dynamics](https://mlsanity.com/paper/2608.12624/) (2026)

## Abstract (arXiv)

Over the past few years, deep learning has risen to the foreground as a topic of massive interest, mainly as a result of successes obtained in solving large-scale image processing tasks. There are multiple challenging mathematical problems involved in applying deep learning: most deep learning methods require the solution of hard optimisation problems, and a good understanding of the tradeoff between computational effort, amount of data and model complexity is required to successfully design a deep learning approach for a given problem. A large amount of progress made in deep learning has been based on heuristic explorations, but there is a growing effort to mathematically understand the structure in existing deep learning methods and to systematically design new deep learning methods to preserve certain types of structure in deep learning. In this article, we review a number of these directions: some deep neural networks can be understood as discretisations of dynamical systems, neural networks can be designed to have desirable properties such as invertibility or group equivariance, and new algorithmic frameworks based on conformal Hamiltonian systems and Riemannian manifolds to solve the optimisation problems have been proposed. We conclude our review of each of these topics by discussing some open problems that we consider to be interesting directions for future research.