# Learning Nonlinear Projections for Reduced-Order Modeling of Dynamical Systems using Constrained Autoencoders

arXiv: https://arxiv.org/abs/2307.15288 · HTML: https://mlsanity.com/paper/2307.15288/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 84 references that are in the map.
- First posted on arXiv 28 Jul 2023.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Analyzing Inverse Problems with Invertible Neural Networks](https://mlsanity.com/paper/1808.04730/) (2018)
- [Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group](https://mlsanity.com/paper/1901.08428/) (2019)
- [A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEs](https://mlsanity.com/paper/2001.04001/) (2020)
- [Group Sparse Regularization for Deep Neural Networks](https://mlsanity.com/paper/1607.00485/) (2016)
- [Reduced order modeling of parametrized systems through autoencoders and SINDy approach: continuation of periodic solutions](https://mlsanity.com/paper/2211.06786/) (2022)
- [Data-Driven Modeling and Prediction of Non-Linearizable Dynamics via Spectral Submanifolds](https://mlsanity.com/paper/2201.04976/) (2022)
- [Optimizing Oblique Projections for Nonlinear Systems using Trajectories](https://mlsanity.com/paper/2106.01211/) (2021)
- [Model Reduction for Nonlinear Systems by Balanced Truncation of State and Gradient Covariance](https://mlsanity.com/paper/2207.14387/) (2022)
- [Balanced Truncation Model Reduction for Lifted Nonlinear Systems](https://mlsanity.com/paper/1907.12084/) (2019)
- [Balanced Truncation Model Order Reduction For Quadratic-Bilinear Control Systems](https://mlsanity.com/paper/1705.00160/) (2017)
- [A quadratic decoder approach to nonintrusive reduced-order modeling of nonlinear dynamical systems](https://mlsanity.com/paper/2209.15412/) (2022)
- [Scalable Computation of Energy Functions for Nonlinear Balanced Truncation](https://mlsanity.com/paper/2209.07645/) (2022)
- [Nonlinear Balanced Truncation: Part 2 -- Model Reduction on Manifolds](https://mlsanity.com/paper/2302.02036/) (2023)

## Built on by (most cited first, 16 of 16)

- [Stability analysis of chaotic systems in latent spaces](https://mlsanity.com/paper/2410.00480/) (2024)
- [A Riemannian Framework for Learning Reduced-order Lagrangian Dynamics](https://mlsanity.com/paper/2410.18868/) (2024)
- [Smooth and Sparse Latent Dynamics in Operator Learning with Jerk Regularization](https://mlsanity.com/paper/2402.15636/) (2024)
- [Data-driven identification of latent port-Hamiltonian systems](https://mlsanity.com/paper/2408.08185/) (2024)
- [GeoHNNs: Geometric Hamiltonian Neural Networks](https://mlsanity.com/paper/2507.15678/) (2025)
- [Toward Adaptive Non-Intrusive Reduced-Order Models: Design and Challenges](https://mlsanity.com/paper/2602.11378/) (2026)
- [Deep Symmetric Autoencoders from the Eckart-Young-Schmidt Perspective](https://mlsanity.com/paper/2506.11641/) (2025)
- [Learning Hamiltonian Dynamics at Scale: A Differential-Geometric Approach](https://mlsanity.com/paper/2509.24627/) (2025)
- [Nonlinear model reduction for transport-dominated problems](https://mlsanity.com/paper/2602.01397/) (2026)
- [History-aware adaptive reduced-order models via incremental singular value decomposition](https://mlsanity.com/paper/2605.28684/) (2026)
- [Nonlinear Model Order Reduction of Dynamical Systems in Process Engineering: Review and Comparison](https://mlsanity.com/paper/2506.12819/) (2025)
- [Context-dependent manifold learning: A neuromodulated constrained autoencoder approach](https://mlsanity.com/paper/2603.11673/) (2026)
- [Deep Invertible Autoencoders for Dimensionality Reduction of Dynamical Systems](https://mlsanity.com/paper/2603.13496/) (2026)
- [Convolutional Symmetric AutoEncoders: enhancing latent stability via differential geometry](https://mlsanity.com/paper/2607.00669/) (2026)
- [Learning Reduced-Order Dynamics with Singularity via Latent-Augmented Neural Ordinary Differential Equations](https://mlsanity.com/paper/2608.22142/) (2026)
- [Periodic Neural Mapping for Unsteady Rotor-Blade Pressure and Aeroelastic Load Prediction](https://mlsanity.com/paper/2609.21590/) (2026)

## Abstract (arXiv)

Recently developed reduced-order modeling techniques aim to approximate nonlinear dynamical systems on low-dimensional manifolds learned from data. This is an effective approach for modeling dynamics in a post-transient regime where the effects of initial conditions and other disturbances have decayed. However, modeling transient dynamics near an underlying manifold, as needed for real-time control and forecasting applications, is complicated by the effects of fast dynamics and nonnormal sensitivity mechanisms. To begin to address these issues, we introduce a parametric class of nonlinear projections described by constrained autoencoder neural networks in which both the manifold and the projection fibers are learned from data. Our architecture uses invertible activation functions and biorthogonal weight matrices to ensure that the encoder is a left inverse of the decoder. We also introduce new dynamics-aware cost functions that promote learning of oblique projection fibers that account for fast dynamics and nonnormality. To demonstrate these methods and the specific challenges they address, we provide a detailed case study of a three-state model of vortex shedding in the wake of a bluff body immersed in a fluid, which has a two-dimensional slow manifold that can be computed analytically. In anticipation of future applications to high-dimensional systems, we also propose several techniques for constructing computationally efficient reduced-order models using our proposed nonlinear projection framework. This includes a novel sparsity-promoting penalty for the encoder that avoids detrimental weight matrix shrinkage via computation on the Grassmann manifold.