# Non-linear manifold ROM with Convolutional Autoencoders and Reduced Over-Collocation method

arXiv: https://arxiv.org/abs/2203.00360 · HTML: https://mlsanity.com/paper/2203.00360/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 57 references that are in the map.
- First posted on arXiv 1 Mar 2022.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data](https://mlsanity.com/paper/2002.12880/) (2020)
- [A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder](https://mlsanity.com/paper/2009.11990/) (2020)
- [POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition](https://mlsanity.com/paper/2101.11845/) (2021)
- [Multi-level Convolutional Autoencoder Networks for Parametric Prediction of Spatio-temporal Dynamics](https://mlsanity.com/paper/1912.11114/) (2019)
- [PDE-based Group Equivariant Convolutional Neural Networks](https://mlsanity.com/paper/2001.09046/) (2020)
- [A Deep Learning approach to Reduced Order Modelling of Parameter Dependent Partial Differential Equations](https://mlsanity.com/paper/2103.06183/) (2021)
- [The Neural Network shifted-Proper Orthogonal Decomposition: a Machine Learning Approach for Non-linear Reduction of Hyperbolic Equations](https://mlsanity.com/paper/2108.06558/) (2021)
- [An artificial neural network approach to bifurcating phenomena in computational fluid dynamics](https://mlsanity.com/paper/2109.10765/) (2021)
- [LQResNet: A Deep Neural Network Architecture for Learning Dynamic Processes](https://mlsanity.com/paper/2103.02249/) (2021)

## Built on by (most cited first, 7 of 7)

- [Neural Galerkin Schemes with Active Learning for High-Dimensional Evolution Equations](https://mlsanity.com/paper/2203.01360/) (2022)
- [A practical existence theorem for reduced order models based on convolutional autoencoders](https://mlsanity.com/paper/2402.00435/) (2024)
- [Generative Adversarial Reduced Order Modelling](https://mlsanity.com/paper/2305.15881/) (2023)
- [A Continuous Convolutional Trainable Filter for Modelling Unstructured Data](https://mlsanity.com/paper/2210.13416/) (2022)
- [Towards a machine learning pipeline in reduced order modelling for inverse problems: neural networks for boundary parametrization, dimensionality reduction and solution manifold approximation](https://mlsanity.com/paper/2210.14764/) (2022)
- [A two stages Deep Learning Architecture for Model Reduction of Parametric Time-Dependent Problems](https://mlsanity.com/paper/2301.09926/) (2023)
- [Hyper-Reduced Autoencoders for Efficient and Accurate Nonlinear Model Reductions](https://mlsanity.com/paper/2303.09630/) (2023)

## Abstract (arXiv)

Non-affine parametric dependencies, nonlinearities and advection-dominated regimes of the model of interest can result in a slow Kolmogorov n-width decay, which precludes the realization of efficient reduced-order models based on linear subspace approximations. Among the possible solutions, there are purely data-driven methods that leverage autoencoders and their variants to learn a latent representation of the dynamical system, and then evolve it in time with another architecture. Despite their success in many applications where standard linear techniques fail, more has to be done to increase the interpretability of the results, especially outside the training range and not in regimes characterized by an abundance of data. Not to mention that none of the knowledge on the physics of the model is exploited during the predictive phase. In order to overcome these weaknesses, we implement the non-linear manifold method introduced by Carlberg et al [37] with hyper-reduction achieved through reduced over-collocation and teacher-student training of a reduced decoder. We test the methodology on a 2d non-linear conservation law and a 2d shallow water models, and compare the results obtained with a purely data-driven method for which the dynamics is evolved in time with a long-short term memory network.