# FINDE: Neural Differential Equations for Finding and Preserving Invariant Quantities

arXiv: https://arxiv.org/abs/2210.00272 · HTML: https://mlsanity.com/paper/2210.00272/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 68 references that are in the map.
- First posted on arXiv 1 Oct 2022.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [WeatherBench: A benchmark dataset for data-driven weather forecasting](https://mlsanity.com/paper/2002.00469/) (2020)
- [Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data](https://mlsanity.com/paper/2002.12880/) (2020)
- [A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups](https://mlsanity.com/paper/2104.09459/) (2021)
- [Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control](https://mlsanity.com/paper/1909.12077/) (2019)
- [Towards Understanding the Spectral Bias of Deep Learning](https://mlsanity.com/paper/1912.01198/) (2019)
- [Symplectic Recurrent Neural Networks](https://mlsanity.com/paper/1909.13334/) (2019)
- [Implicit Gradient Regularization](https://mlsanity.com/paper/2009.11162/) (2020)
- [Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/1910.05266/) (2019)
- [Learning to Control PDEs with Differentiable Physics](https://mlsanity.com/paper/2001.07457/) (2020)
- [Learning Stable Deep Dynamics Models](https://mlsanity.com/paper/2001.06116/) (2020)
- [Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints](https://mlsanity.com/paper/2010.13581/) (2020)
- [Automatic Symmetry Discovery with Lie Algebra Convolutional Network](https://mlsanity.com/paper/2109.07103/) (2021)
- [Dissipative SymODEN: Encoding Hamiltonian Dynamics with Dissipation and Control into Deep Learning](https://mlsanity.com/paper/2002.08860/) (2020)
- [AI Poincar'e: Machine Learning Conservation Laws from Trajectories](https://mlsanity.com/paper/2011.04698/) (2020)
- [NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data](https://mlsanity.com/paper/1908.03190/) (2019)
- [Learning Poisson systems and trajectories of autonomous systems via Poisson neural networks](https://mlsanity.com/paper/2012.03133/) (2020)
- [Improved Generalization Bounds of Group Invariant / Equivariant Deep Networks via Quotient Feature Spaces](https://mlsanity.com/paper/1910.06552/) (2019)
- [Universal Approximation Property of Neural Ordinary Differential Equations](https://mlsanity.com/paper/2012.02414/) (2020)
- [Which priors matter? Benchmarking models for learning latent dynamics](https://mlsanity.com/paper/2111.05458/) (2021)
- [Noether Networks: Meta-Learning Useful Conserved Quantities](https://mlsanity.com/paper/2112.03321/) (2021)
- [Learning Physical Constraints with Neural Projections](https://mlsanity.com/paper/2006.12745/) (2020)

## Built on by (most cited first, 12 of 12)

- [Physics-Constrained Flow Matching: Sampling Generative Models with Hard Constraints](https://mlsanity.com/paper/2506.04171/) (2025)
- [Parallelizing non-linear sequential models over the sequence length](https://mlsanity.com/paper/2309.12252/) (2023)
- [End-to-End Probabilistic Framework for Learning with Hard Constraints](https://mlsanity.com/paper/2506.07003/) (2025)
- [Structure-Preserving Physics-Informed Neural Networks With Energy or Lyapunov Structure](https://mlsanity.com/paper/2401.04986/) (2024)
- [Projected Neural Differential Equations for Learning Constrained Dynamics](https://mlsanity.com/paper/2410.23667/) (2024)
- [Learning Dynamical Systems from Noisy Data with Inverse-Explicit Integrators](https://mlsanity.com/paper/2306.03548/) (2023)
- [Discovering Phase Space Structure in Learned Hamiltonian Systems](https://mlsanity.com/paper/2509.23003/) (2025)
- [Pseudo-Hamiltonian system identification](https://mlsanity.com/paper/2305.06920/) (2023)
- [Poisson-Dirac Neural Networks for Modeling Coupled Dynamical Systems across Domains](https://mlsanity.com/paper/2410.11480/) (2024)
- [An Invariant Compiler for Neural ODEs in AI-Accelerated Scientific Simulation](https://mlsanity.com/paper/2603.23861/) (2026)
- [Learning partially observed systems with neural Hamiltonian ordinary differential equations](https://mlsanity.com/paper/2605.23510/) (2026)
- [OrthoReg: Orthogonal Regularization for Hybrid Symbolic-Neural Dynamical Systems](https://mlsanity.com/paper/2606.19145/) (2026)

## Abstract (arXiv)

Many real-world dynamical systems are associated with first integrals (a.k.a. invariant quantities), which are quantities that remain unchanged over time. The discovery and understanding of first integrals are fundamental and important topics both in the natural sciences and in industrial applications. First integrals arise from the conservation laws of system energy, momentum, and mass, and from constraints on states; these are typically related to specific geometric structures of the governing equations. Existing neural networks designed to ensure such first integrals have shown excellent accuracy in modeling from data. However, these models incorporate the underlying structures, and in most situations where neural networks learn unknown systems, these structures are also unknown. This limitation needs to be overcome for scientific discovery and modeling of unknown systems. To this end, we propose first integral-preserving neural differential equation (FINDE). By leveraging the projection method and the discrete gradient method, FINDE finds and preserves first integrals from data, even in the absence of prior knowledge about underlying structures. Experimental results demonstrate that FINDE can predict future states of target systems much longer and find various quantities consistent with well-known first integrals in a unified manner.