# Data-Augmented Predictive Deep Neural Network: Enhancing the extrapolation capabilities of non-intrusive surrogate models

arXiv: https://arxiv.org/abs/2410.13376 · HTML: https://mlsanity.com/paper/2410.13376/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Linearly-Recurrent Autoencoder Networks for Learning Dynamics](https://mlsanity.com/paper/1712.01378/) (2017)
- [Transformers for Modeling Physical Systems](https://mlsanity.com/paper/2010.03957/) (2020)
- [A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEs](https://mlsanity.com/paper/2001.04001/) (2020)
- [POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition](https://mlsanity.com/paper/2101.11845/) (2021)
- [β-Variational autoencoders and transformers for reduced-order modelling of fluid flows](https://mlsanity.com/paper/2304.03571/) (2023)
- [Multi-level Convolutional Autoencoder Networks for Parametric Prediction of Spatio-temporal Dynamics](https://mlsanity.com/paper/1912.11114/) (2019)
- [Reduced order modeling of parametrized systems through autoencoders and SINDy approach: continuation of periodic solutions](https://mlsanity.com/paper/2211.06786/) (2022)
- [A Comprehensive Review of Latent Space Dynamics Identification Algorithms for Intrusive and Non-Intrusive Reduced-Order-Modeling](https://mlsanity.com/paper/2403.10748/) (2024)
- [Long-time prediction of nonlinear parametrized dynamical systems by deep learning-based reduced order models](https://mlsanity.com/paper/2201.10215/) (2022)
- [Active-Learning-Driven Surrogate Modeling for Efficient Simulation of Parametric Nonlinear Systems](https://mlsanity.com/paper/2306.06174/) (2023)

## Built on by (most cited first, 3 of 3)

- [Spatially-Enhanced Temporal Fusion Transformer: Interpretable Multi-Output Prediction for Parametric Dynamical Systems with Time-Varying Inputs](https://mlsanity.com/paper/2505.00473/) (2025)
- [Drivetrain simulation using variational autoencoders](https://mlsanity.com/paper/2501.17653/) (2025)
- [Multi-Fidelity Learning with Shallow Recurrent Decoders for Multi-Physics Applications](https://mlsanity.com/paper/2606.05202/) (2026)

## Abstract (arXiv)

Numerically solving a large parametric nonlinear dynamical system is challenging due to its high complexity and the high computational costs. In recent years, machine-learning-aided surrogates are being actively researched. However, many methods fail in accurately generalizing in the entire time interval [0, T], when the training data is available only in a training time interval [0, T0], with T0<T. To improve the extrapolation capabilities of the surrogate models in the entire time domain, we propose a new deep learning framework, where kernel dynamic mode decomposition (KDMD) is employed to evolve the dynamics of the latent space generated by the encoder part of a convolutional autoencoder (CAE). After adding the KDMD-decoder-extrapolated data into the original data set, we train the CAE along with a feed-forward deep neural network using the augmented data. The trained network can predict future states outside the training time interval at any out-of-training parameter samples. The proposed method is tested on two numerical examples: a FitzHugh-Nagumo model and a model of incompressible flow past a cylinder. Numerical results show accurate and fast prediction performance in both the time and the parameter domain.