# Comparison of neural network training strategies for the simulation of dynamical systems

arXiv: https://arxiv.org/abs/2512.03851 · HTML: https://mlsanity.com/paper/2512.03851/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 3 Dec 2025.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Tune: A Research Platform for Distributed Model Selection and Training](https://mlsanity.com/paper/1807.05118/) (2018)
- [Transformers for Modeling Physical Systems](https://mlsanity.com/paper/2010.03957/) (2020)
- [Deep networks for system identification: a Survey](https://mlsanity.com/paper/2301.12832/) (2023)
- [Constructing Neural Network-Based Models for Simulating Dynamical Systems](https://mlsanity.com/paper/2111.01495/) (2021)
- [Capacity and Trainability in Recurrent Neural Networks](https://mlsanity.com/paper/1611.09913/) (2016)
- [Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking](https://mlsanity.com/paper/2206.03305/) (2022)
- [Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation](https://mlsanity.com/paper/2504.19322/) (2025)
- [Distributionally Robust Policy Learning under Concept Drifts](https://mlsanity.com/paper/2412.14297/) (2024)
- ["Parallel Training Considered Harmful?": Comparing series-parallel and parallel feedforward network training](https://mlsanity.com/paper/1706.07119/) (2017)
- [Introducing a Deep Neural Network-based Model Predictive Control Framework for Rapid Controller Implementation](https://mlsanity.com/paper/2310.08392/) (2023)
- [Data-driven identification of nonlinear dynamical systems with LSTM autoencoders and Normalizing Flows](https://mlsanity.com/paper/2503.03977/) (2025)

## Abstract (arXiv)

Neural networks have become a widely adopted tool for modeling nonlinear dynamical systems from data. However, the choice of training strategy remains a key design decision, particularly for simulation tasks. This paper compares two predominant strategies: parallel and series-parallel training. The conducted empirical analysis spans five neural network architectures and two examples: a pneumatic valve test bench and an industrial robot benchmark. The study reveals that, even though series-parallel training dominates current practice, parallel training consistently yields better long-term prediction accuracy. Additionally, this work clarifies the often inconsistent terminology in the literature and relate both strategies to concepts from system identification. The findings suggest that parallel training should be considered the default training strategy for neural network-based simulation of dynamical systems.