# A Comparison of Neural Network Architectures for Data-Driven Reduced-Order Modeling

arXiv: https://arxiv.org/abs/2110.03442 · HTML: https://mlsanity.com/paper/2110.03442/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 5 Oct 2021.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [Simple and Deep Graph Convolutional Networks](https://mlsanity.com/paper/2007.02133/) (2020)
- [Residual Gated Graph ConvNets](https://mlsanity.com/paper/1711.07553/) (2017)
- [A guide to convolution arithmetic for deep learning](https://mlsanity.com/paper/1603.07285/) (2016)
- [A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEs](https://mlsanity.com/paper/2001.04001/) (2020)
- [Deep Neural Networks for Nonlinear Model Order Reduction of Unsteady Flows](https://mlsanity.com/paper/2007.00936/) (2020)
- [PAN: Path Integral Based Convolution for Deep Graph Neural Networks](https://mlsanity.com/paper/1904.10996/) (2019)

## Built on by (most cited first, 22 of 22)

- [A graph convolutional autoencoder approach to model order reduction for parametrized PDEs](https://mlsanity.com/paper/2305.08573/) (2023)
- [Super-Resolution Analysis via Machine Learning: A Survey for Fluid Flows](https://mlsanity.com/paper/2301.10937/) (2023)
- [Deep Learning in Deterministic Computational Mechanics](https://mlsanity.com/paper/2309.15421/) (2023)
- [RecFNO: a resolution-invariant flow and heat field reconstruction method from sparse observations via Fourier neural operator](https://mlsanity.com/paper/2302.09808/) (2023)
- [Multiscale Graph Neural Network Autoencoders for Interpretable Scientific Machine Learning](https://mlsanity.com/paper/2302.06186/) (2023)
- [Physics-aware Reduced-order Modeling of Transonic Flow via β-Variational Autoencoder](https://mlsanity.com/paper/2205.00608/) (2022)
- [Canonical and Noncanonical Hamiltonian Operator Inference](https://mlsanity.com/paper/2304.06262/) (2023)
- [GFN: A graph feedforward network for resolution-invariant reduced operator learning in multifidelity applications](https://mlsanity.com/paper/2406.03569/) (2024)
- [Latent assimilation with implicit neural representations for unknown dynamics](https://mlsanity.com/paper/2309.09574/) (2023)
- [Generative Adversarial Reduced Order Modelling](https://mlsanity.com/paper/2305.15881/) (2023)
- [Low-dimensional Data-based Surrogate Model of a Continuum-mechanical Musculoskeletal System Based on Non-intrusive Model Order Reduction](https://mlsanity.com/paper/2302.06528/) (2023)
- [An AI-based Domain-Decomposition Non-Intrusive Reduced-Order Model for Extended Domains applied to Multiphase Flow in Pipes](https://mlsanity.com/paper/2202.06170/) (2022)
- [AI-enhanced iterative solvers for accelerating the solution of large scale parametrized systems](https://mlsanity.com/paper/2207.02543/) (2022)
- [Data-driven identification of latent port-Hamiltonian systems](https://mlsanity.com/paper/2408.08185/) (2024)
- [A physics-informed neural network method for the approximation of slow invariant manifolds for the general class of stiff systems of ODEs](https://mlsanity.com/paper/2403.11591/) (2024)
- [Hierarchical-embedding autoencoder with a predictor (HEAP) as efficient architecture for learning long-term evolution of complex multi-scale physical systems](https://mlsanity.com/paper/2505.18857/) (2025)
- [Time Extrapolation with Graph Convolutional Autoencoder and Tensor Train Decomposition](https://mlsanity.com/paper/2511.23037/) (2025)
- [Surrogates for Physics-based and Data-driven Modelling of Parametric Systems: Review and New Perspectives](https://mlsanity.com/paper/2603.12870/) (2026)
- [Multi-Hierarchical Surrogate Learning for Structural Dynamical Crash Simulations Using Graph Convolutional Neural Networks](https://mlsanity.com/paper/2402.09234/) (2024)
- [Nonlinear model reduction for transport-dominated problems](https://mlsanity.com/paper/2602.01397/) (2026)
- [NN-OpInf: an operator inference approach using structure-preserving composable neural networks](https://mlsanity.com/paper/2603.08488/) (2026)
- [Deep Wave Network for Modeling Multi-Scale Physical Dynamics](https://mlsanity.com/paper/2605.04198/) (2026)

## Abstract (arXiv)

The popularity of deep convolutional autoencoders (CAEs) has engendered new and effective reduced-order models (ROMs) for the simulation of large-scale dynamical systems. Despite this, it is still unknown whether deep CAEs provide superior performance over established linear techniques or other network-based methods in all modeling scenarios. To elucidate this, the effect of autoencoder architecture on its associated ROM is studied through the comparison of deep CAEs against two alternatives: a simple fully connected autoencoder, and a novel graph convolutional autoencoder. Through benchmark experiments, it is shown that the superior autoencoder architecture for a given ROM application is highly dependent on the size of the latent space and the structure of the snapshot data, with the proposed architecture demonstrating benefits on data with irregular connectivity when the latent space is sufficiently large.