mlsanityData to 7 Oct 2026

β-Variational autoencoders and transformers for reduced-order modelling of fluid flows

arXiv 2304.03571 ↗First posted 7 Apr 2023physics.flu-dyn cs.LGcited by 76

Not on the map. Why?

Abstract (arXiv)

Variational autoencoder (VAE) architectures have the potential to develop reduced-order models (ROMs) for chaotic fluid flows. We propose a method for learning compact and near-orthogonal ROMs using a combination of a β-VAE and a transformer, tested on numerical data from a two-dimensional viscous flow in both periodic and chaotic regimes. The β-VAE is trained to learn a compact latent representation of the flow velocity, and the transformer is trained to predict the temporal dynamics in latent space. Using the β-VAE to learn disentangled representations in latent-space, we obtain a more interpretable flow model with features that resemble those observed in the proper orthogonal decomposition, but with a more efficient representation. Using Poincar'e maps, the results show that our method can capture the underlying dynamics of the flow outperforming other prediction models. The proposed method has potential applications in other fields such as weather forecasting, structural dynamics or biomedical engineering.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). Builds on 22 of its 69 references that are in the map. First posted on arXiv 7 Apr 2023.

Builds on

22 of 69 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  6. 2013Auto-Encoding Variational Bayescited by 3,980
  7. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  8. 2021Transformers in Vision: A Surveycited by 1,009
  9. 2021AST: Audio Spectrogram Transformercited by 634
  10. 2019Machine Learning for Fluid Mechanicscited by 380
  11. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  12. 2019Lookahead Optimizer: k steps forward, 1 step backcited by 333
10 more
  1. 2018Understanding disentangling in β-VAEcited by 332
  2. 2019Time2Vec: Learning a Vector Representation of Timecited by 208
  3. 2021Enhancing Computational Fluid Dynamics with Machine Learningcited by 137
  4. 2020Transformers for Modeling Physical Systemscited by 105
  5. 2020A Survey of Deep Learning Techniques for Neural Machine Translationcited by 50
  6. 2021Towards extraction of orthogonal and parsimonious non-linear modes from turbulent flowscited by 44
  7. 2020Deep Neural Networks for Nonlinear Model Order Reduction of Unsteady Flowscited by 36
  8. 2020Image Captioning through Image Transformercited by 29
  9. 2019Data-Driven POD-Galerkin Reduced Order Model for Turbulent Flowscited by 21
  10. 2022Predicting the temporal dynamics of turbulent channels through deep learningcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 16; 2025: 20; 2026: 32.

76 papers.

Sort:

By area:

  1. 2025Sparse identification of nonlinear dynamics and Koopman operators with Shallow Recurrent Decoder Networkscited by 19
  2. 2023Multi-scale Time-stepping of Partial Differential Equations with Transformerscited by 14
  3. 2025MORPH: PDE Foundation Models with Arbitrary Data Modalitycited by 12
  4. 2023Stochastic Latent Transformer: Efficient Modelling of Stochastically Forced Zonal Jetscited by 8
  5. 2024VENI, VINDy, VICI: a generative reduced-order modeling framework with uncertainty quantificationcited by 8
  6. 2025Diff-SPORT: Diffusion-based Sensor Placement Optimization and Reconstruction of Turbulent flows in urban environmentscited by 8
  7. 2023Generative Adversarial Reduced Order Modellingcited by 7
  8. 2024A conditional latent autoregressive recurrent model for generation and forecasting of beam dynamics in particle acceleratorscited by 7
  9. 2025Generative emulation of chaotic dynamics with coherent priorcited by 7
  10. 2024Generative prediction of flow fields around an obstacle using the diffusion modelcited by 6
  11. 2024Time-inversion of spatiotemporal beam dynamics using uncertainty-aware latent evolution reversalcited by 6
  12. 2024Towards aerodynamic surrogate modeling based on β-variational autoencoderscited by 5
38 more
  1. 2024Reduced-order modeling of unsteady fluid flow using neural network ensemblescited by 4
  2. 2024Enhancing Graph U-Nets for Mesh-Agnostic Spatio-Temporal Flow Predictioncited by 4
  3. 2026Explainable AI: Learning from the Learnerscited by 4
  4. 2023Data-driven Nonlinear Parametric Model Order Reduction Framework using Deep Hierarchical Variational Autoencodercited by 3
  5. 2023Branched Latent Neural Mapscited by 3
  6. 2023Easy attention: A simple attention mechanism for temporal predictions with transformerscited by 3
  7. 2023Volume-Preserving Transformers for Learning Time Series Data with Structurecited by 3
  8. 2024Decoder Decomposition for the Analysis of the Latent Space of Nonlinear Autoencoders With Wind-Tunnel Experimental Datacited by 3
  9. 2024Decoding complexity: how machine learning is redefining scientific discoverycited by 3
  10. 2024Data-Augmented Predictive Deep Neural Network: Enhancing the extrapolation capabilities of non-intrusive surrogate modelscited by 3
  11. 2024Model-Agnostic AI Framework with Explicit Time Integration for Long-Term Fluid Dynamics Predictioncited by 3
  12. 2025A Deep Learning approach for parametrized and time dependent Partial Differential Equations using Dimensionality Reduction and Neural ODEscited by 3
  13. 2026Adaptive Diffusion Posterior Sampling for Data and Model Fusion of Complex Nonlinear Dynamical Systemscited by 3
  14. 2024A physics-informed neural network method for the approximation of slow invariant manifolds for the general class of stiff systems of ODEscited by 2
  15. 2025Disentangled Latent Spaces for Reduced Order Models using Deterministic Autoencoderscited by 2
  16. 2025Regional climate projections using a deep-learning-based model-ranking and downscaling framework: Application to European climate zonescited by 2
  17. 2025Physics-informed Reduced Order Modeling of Time-dependent PDEs via Differentiable Solverscited by 2
  18. 2025Convolutional autoencoders for the reconstruction of three-dimensional interfacial multiphase flowscited by 2
  19. 2025Conditional neural field for spatial dimension reduction of turbulence data: a comparison studycited by 2
  20. 2026AeroJEPA: Learning Semantic Latent Representations for Scalable 3D Aerodynamic Field Modelingcited by 2
  21. 2023Dimensionality Reduction and Dynamical Mode Recognition of Circular Arrays of Flame Oscillators Using Deep Neural Networkcited by 1
  22. 2024Beyond the Kolmogorov Barrier: A Learnable Weighted Hybrid Autoencoder for Model Order Reductioncited by 1
  23. 2025Capturing Extreme Events in Turbulence using an Extreme Variational Autoencoder (xVAE)cited by 1
  24. 2025Spatially-Enhanced Temporal Fusion Transformer: Interpretable Multi-Output Prediction for Parametric Dynamical Systems with Time-Varying Inputscited by 1
  25. 2025Variational Rank Reduction Autoencoders for Generative Thermal Designcited by 1
  26. 2026Data-Driven Reduced-Complexity Modeling of Fluid Flows: A Community Challengecited by 1
  27. 2026LLM4Fluid: Large Language Models as Generalizable Neural Solvers for Fluid Dynamicscited by 1
  28. 2024Mixing Artificial and Natural Intelligence: From Statistical Mechanics to AI and Back to Turbulencecited by 0
  29. 2024Data-driven low-dimensional model of a sedimenting flexible fibercited by 0
  30. 2024Parametric Taylor series based latent dynamics identification neural networkscited by 0
  31. 2025UP-dROM : Uncertainty-Aware and Parametrised dynamic Reduced-Order Model, application to unsteady flowscited by 0
  32. 2025Fourier-Invertible Neural Encoder (FINE) for Homogeneous Flowscited by 0
  33. 2025Blending data and physics for reduced-order modeling of systems with spatiotemporal chaotic dynamicscited by 0
  34. 2025A DeepONet joint Neural Tangent Kernel Hybrid Framework for Physics-Informed Inverse Source Problems and Robust Image Reconstructioncited by 0
  35. 2025Physically Interpretable Representation Learning with Gaussian Mixture Variational AutoEncoder (GM-VAE)cited by 0
  36. 2025Can Transformers overcome the lack of data in the simulation of history-dependent flows?cited by 0
  37. 2025Reduced Order Modeling for Tsunami Forecasting with Bayesian Hierarchical Poolingcited by 0
  38. 2026Coupled Diffusion-Encoder Models for Reconstruction of Flow Fieldscited by 0

50 most cited of 76 All 76 (JSON)