# NLP Inspired Training Mechanics For Modeling Transient Dynamics

arXiv: https://arxiv.org/abs/2211.02716 · HTML: https://mlsanity.com/paper/2211.02716/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 14 references that are in the map.
- First posted on arXiv 4 Nov 2022.

## Builds on (most cited first, 11 of 11)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/1801.06879/) (2018)
- [Prediction of Aerodynamic Flow Fields Using Convolutional Neural Networks](https://mlsanity.com/paper/1905.13166/) (2019)
- [EikoNet: Solving the Eikonal equation with Deep Neural Networks](https://mlsanity.com/paper/2004.00361/) (2020)
- [DiscretizationNet: A Machine-Learning based solver for Navier-Stokes Equations using Finite Volume Discretization](https://mlsanity.com/paper/2005.08357/) (2020)
- [Unsupervised Deep Learning Algorithm for PDE-based Forward and Inverse Problems](https://mlsanity.com/paper/1904.05417/) (2019)
- [Multiscale Neural Operator: Learning Fast and Grid-independent PDE Solvers](https://mlsanity.com/paper/2207.11417/) (2022)
- [A composable machine-learning approach for steady-state simulations on high-resolution grids](https://mlsanity.com/paper/2210.05837/) (2022)
- [Stochastic Adversarial Koopman Model for Dynamical Systems](https://mlsanity.com/paper/2109.05095/) (2021)

## Built on by (most cited first, 1 of 1)

- [A domain decomposition-based autoregressive deep learning model for unsteady and nonlinear partial differential equations](https://mlsanity.com/paper/2408.14461/) (2024)

## Abstract (arXiv)

In recent years, Machine learning (ML) techniques developed for Natural Language Processing (NLP) have permeated into developing better computer vision algorithms. In this work, we use such NLP-inspired techniques to improve the accuracy, robustness and generalizability of ML models for simulating transient dynamics. We introduce teacher forcing and curriculum learning based training mechanics to model vortical flows and show an enhancement in accuracy for ML models, such as FNO and UNet by more than 50%.