# A Simultaneous Approach for Training Neural Differential-Algebraic Systems of Equations

arXiv: https://arxiv.org/abs/2504.04665 · HTML: https://mlsanity.com/paper/2504.04665/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 65 references that are in the map.
- First posted on arXiv 7 Apr 2025.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [Universal Differential Equations for Scientific Machine Learning](https://mlsanity.com/paper/2001.04385/) (2020)
- [On Neural Differential Equations](https://mlsanity.com/paper/2202.02435/) (2022)
- [Equinox: neural networks in JAX via callable PyTrees and filtered transformations](https://mlsanity.com/paper/2111.00254/) (2021)
- [OMLT: Optimization & Machine Learning Toolkit](https://mlsanity.com/paper/2202.02414/) (2022)
- [DAE-PINN: A Physics-Informed Neural Network Model for Simulating Differential-Algebraic Equations with Application to Power Networks](https://mlsanity.com/paper/2109.04304/) (2021)
- [Constrained Neural Ordinary Differential Equations with Stability Guarantees](https://mlsanity.com/paper/2004.10883/) (2020)
- [MINN: Learning the dynamics of differential-algebraic equations and application to battery modeling](https://mlsanity.com/paper/2304.14422/) (2023)
- [Comparison of neural closure models for discretised PDEs](https://mlsanity.com/paper/2210.14675/) (2022)
- [Learning Neural Differential Algebraic Equations via Operator Splitting](https://mlsanity.com/paper/2403.12938/) (2024)
- [Projected Neural Differential Equations for Learning Constrained Dynamics](https://mlsanity.com/paper/2410.23667/) (2024)
- [A Comparison of Strategies to Embed Physics-Informed Neural Networks in Nonlinear Model Predictive Control Formulations Solved via Direct Transcription](https://mlsanity.com/paper/2501.06335/) (2025)
- [Learning To Solve Differential Equation Constrained Optimization Problems](https://mlsanity.com/paper/2410.01786/) (2024)
- [Training Neural ODEs Using Fully Discretized Simultaneous Optimization](https://mlsanity.com/paper/2502.15642/) (2025)
- [A Unifying Modeling Abstraction for Infinite-Dimensional Optimization](https://mlsanity.com/paper/2106.12689/) (2021)
- [Neural Port-Hamiltonian Differential Algebraic Equations for Compositional Learning of Electrical Networks](https://mlsanity.com/paper/2412.11215/) (2024)
- [Physical Information Neural Networks for Solving High-index Differential-algebraic Equation Systems Based on Radau Methods](https://mlsanity.com/paper/2310.12846/) (2023)
- [Formulations and scalability of neural network surrogates in nonlinear optimization problems](https://mlsanity.com/paper/2412.11403/) (2024)
- [Event Constrained Programming](https://mlsanity.com/paper/2501.06353/) (2025)

## Built on by (most cited first, 4 of 4)

- [Physics-Constrained Machine Learning for Chemical Engineering](https://mlsanity.com/paper/2508.20649/) (2025)
- [Physics-Guided Dimension Reduction for Simulation-Free Operator Learning of Stiff Differential-Algebraic Systems](https://mlsanity.com/paper/2604.19930/) (2026)
- [Efficient Training of Physics-enhanced Neural ODEs via Direct Collocation and Nonlinear Programming](https://mlsanity.com/paper/2505.03552/) (2025)
- [A tale of perfect fit and phantom optima: how data-driven models can fail in real-time optimization](https://mlsanity.com/paper/2608.23885/) (2026)

## Abstract (arXiv)

Scientific machine learning is an emerging field that broadly describes the combination of scientific computing and machine learning to address challenges in science and engineering. Within the context of differential equations, this has produced highly influential methods, such as neural ordinary differential equations (NODEs). Recent works extend this line of research to consider neural differential-algebraic systems of equations (DAEs), where some unknown relationships within the DAE are learned from data. Training neural DAEs, similarly to neural ODEs, is computationally expensive, as it requires the solution of a DAE for every parameter update. Further, the rigorous consideration of algebraic constraints is difficult within common deep learning training algorithms such as stochastic gradient descent. In this work, we apply the simultaneous approach to neural DAE problems, resulting in a fully discretized nonlinear optimization problem, which is solved to local optimality and simultaneously obtains the neural network parameters and the solution to the corresponding DAE. We extend recent work demonstrating the simultaneous approach for neural ODEs, by presenting a general framework to solve neural DAEs, with explicit consideration of hybrid models, where some components of the DAE are known, e.g. physics-informed constraints. Furthermore, we present a general strategy for improving the performance and convergence of the nonlinear programming solver, based on solving an auxiliary problem for initialization and approximating Hessian terms. We achieve promising results in terms of accuracy, model generalizability and computational cost, across different problem settings such as sparse data, unobserved states and multiple trajectories. Lastly, we provide several promising future directions to improve the scalability and robustness of our approach.