# Constrained Block Nonlinear Neural Dynamical Models

arXiv: https://arxiv.org/abs/2101.01864 · HTML: https://mlsanity.com/paper/2101.01864/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 45 references that are in the map.
- First posted on arXiv 6 Jan 2021.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [Structured Inference Networks for Nonlinear State Space Models](https://mlsanity.com/paper/1609.09869/) (2016)
- [Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems](https://mlsanity.com/paper/1708.06850/) (2017)
- [Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems](https://mlsanity.com/paper/1801.01236/) (2018)
- [Nonlinear System Identification: A User-Oriented Roadmap](https://mlsanity.com/paper/1902.00683/) (2019)
- [Long Short-Term Memory Kalman Filters:Recurrent Neural Estimators for Pose Regularization](https://mlsanity.com/paper/1708.01885/) (2017)
- [Stabilizing Gradients for Deep Neural Networks via Efficient SVD Parameterization](https://mlsanity.com/paper/1803.09327/) (2018)
- [IMEXnet: A Forward Stable Deep Neural Network](https://mlsanity.com/paper/1903.02639/) (2019)

## Built on by (most cited first, 9 of 9)

- [Constructing Neural Network-Based Models for Simulating Dynamical Systems](https://mlsanity.com/paper/2111.01495/) (2021)
- [Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems](https://mlsanity.com/paper/2306.13867/) (2023)
- [Neural Ordinary Differential Equations for Nonlinear System Identification](https://mlsanity.com/paper/2203.00120/) (2022)
- [Learning Neural Differential Algebraic Equations via Operator Splitting](https://mlsanity.com/paper/2403.12938/) (2024)
- [Structural Inference of Networked Dynamical Systems with Universal Differential Equations](https://mlsanity.com/paper/2207.04962/) (2022)
- [Synthetic Time Series Forecasting with Transformer Architectures: Extensive Simulation Benchmarks](https://mlsanity.com/paper/2505.20048/) (2025)
- [Learnable Koopman-Enhanced Transformer-Based Time Series Forecasting with Spectral Control](https://mlsanity.com/paper/2602.02592/) (2026)
- [Domain-aware Control-oriented Neural Models for Autonomous Underwater Vehicles](https://mlsanity.com/paper/2208.07333/) (2022)
- [DeepKoopFormer: A Koopman Enhanced Transformer Based Architecture for Time Series Forecasting](https://mlsanity.com/paper/2508.02616/) (2025)

## Abstract (arXiv)

Neural network modules conditioned by known priors can be effectively trained and combined to represent systems with nonlinear dynamics. This work explores a novel formulation for data-efficient learning of deep control-oriented nonlinear dynamical models by embedding local model structure and constraints. The proposed method consists of neural network blocks that represent input, state, and output dynamics with constraints placed on the network weights and system variables. For handling partially observable dynamical systems, we utilize a state observer neural network to estimate the states of the system's latent dynamics. We evaluate the performance of the proposed architecture and training methods on system identification tasks for three nonlinear systems: a continuous stirred tank reactor, a two tank interacting system, and an aerodynamics body. Models optimized with a few thousand system state observations accurately represent system dynamics in open loop simulation over thousands of time steps from a single set of initial conditions. Experimental results demonstrate an order of magnitude reduction in open-loop simulation mean squared error for our constrained, block-structured neural models when compared to traditional unstructured and unconstrained neural network models.