# Accelerating Dynamical System Simulations with Contracting and Physics-Projected Neural-Newton Solvers

arXiv: https://arxiv.org/abs/2106.02543 · HTML: https://mlsanity.com/paper/2106.02543/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 50 references that are in the map.
- First posted on arXiv 4 Jun 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations](https://mlsanity.com/paper/1710.10121/) (2017)
- [Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit](https://mlsanity.com/paper/1905.09883/) (2019)
- [Graph Neural Ordinary Differential Equations](https://mlsanity.com/paper/1911.07532/) (2019)
- [Dissecting Neural ODEs](https://mlsanity.com/paper/2002.08071/) (2020)
- [Multiscale Deep Equilibrium Models](https://mlsanity.com/paper/2006.08656/) (2020)
- [Data Driven Governing Equations Approximation Using Deep Neural Networks](https://mlsanity.com/paper/1811.05537/) (2018)
- [Physics-Informed Neural Networks for Power Systems](https://mlsanity.com/paper/1911.03737/) (2019)
- [Deep Learning via Dynamical Systems: An Approximation Perspective](https://mlsanity.com/paper/1912.10382/) (2019)
- [Stable Recurrent Models](https://mlsanity.com/paper/1805.10369/) (2018)
- [Hierarchical Deep Learning of Multiscale Differential Equation Time-Steppers](https://mlsanity.com/paper/2008.09768/) (2020)
- [Modeling System Dynamics with Physics-Informed Neural Networks Based on Lagrangian Mechanics](https://mlsanity.com/paper/2005.14617/) (2020)
- [On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs](https://mlsanity.com/paper/2004.01806/) (2020)
- [Contracting Implicit Recurrent Neural Networks: Stable Models with Improved Trainability](https://mlsanity.com/paper/1912.10402/) (2019)
- [Long-time predictive modeling of nonlinear dynamical systems using neural networks](https://mlsanity.com/paper/1805.12547/) (2018)
- [ANODEV2: A Coupled Neural ODE Evolution Framework](https://mlsanity.com/paper/1906.04596/) (2019)
- [Emulating AC OPF solvers for Obtaining Sub-second Feasible, Near-Optimal Solutions](https://mlsanity.com/paper/2012.10031/) (2020)

## Built on by (most cited first, 1 of 1)

- [A Unified Approach for Learning the Dynamics of Power System Generators and Inverter-based Resources](https://mlsanity.com/paper/2409.14454/) (2024)

## Abstract (arXiv)

Recent advances in deep learning have allowed neural networks (NNs) to successfully replace traditional numerical solvers in many applications, thus enabling impressive computing gains. One such application is time domain simulation, which is indispensable for the design, analysis and operation of many engineering systems. Simulating dynamical systems with implicit Newton-based solvers is a computationally heavy task, as it requires the solution of a parameterized system of differential and algebraic equations at each time step. A variety of NN-based methodologies have been shown to successfully approximate the trajectories computed by numerical solvers at a fraction of the time. However, few previous works have used NNs to model the numerical solver itself. For the express purpose of accelerating time domain simulation speeds, this paper proposes and explores two complementary alternatives for modeling numerical solvers. First, we use a NN to mimic the linear transformation provided by the inverse Jacobian in a single Newton step. Using this procedure, we evaluate and project the exact, physics-based residual error onto the NN mapping, thus leaving physics ``in the loop''. The resulting tool, termed the Physics-pRojected Neural-Newton Solver (PRoNNS), is able to achieve an extremely high degree of numerical accuracy at speeds which were observed to be up to 31% faster than a Newton-based solver. In the second approach, we model the Newton solver at the heart of an implicit Runge-Kutta integrator as a contracting map iteratively seeking a fixed point on a time domain trajectory. The associated recurrent NN simulation tool, termed the Contracting Neural-Newton Solver (CoNNS), is embedded with training constraints (via CVXPY Layers) which guarantee the mapping provided by the NN satisfies the Banach fixed-point theorem.