# Reduced order modeling for flow and transport problems with Barlow Twins self-supervised learning

arXiv: https://arxiv.org/abs/2202.05460 · HTML: https://mlsanity.com/paper/2202.05460/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 76 references that are in the map.
- First posted on arXiv 11 Feb 2022.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEs](https://mlsanity.com/paper/2001.04001/) (2020)
- [A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder](https://mlsanity.com/paper/2009.11990/) (2020)
- [A framework for data-driven solution and parameter estimation of PDEs using conditional generative adversarial networks](https://mlsanity.com/paper/2105.13136/) (2021)
- [Non-intrusive reduced order modeling of natural convection in porous media using convolutional autoencoders: comparison with linear subspace techniques](https://mlsanity.com/paper/2107.11460/) (2021)
- [Physics-informed Neural Networks for Solving Nonlinear Diffusivity and Biot's equations](https://mlsanity.com/paper/2002.08235/) (2020)
- [Efficient nonlinear manifold reduced order model](https://mlsanity.com/paper/2011.07727/) (2020)
- [Nonlinear proper orthogonal decomposition for convection-dominated flows](https://mlsanity.com/paper/2110.08295/) (2021)
- [Non-intrusive reduced order modeling of poroelasticity of heterogeneous media based on a discontinuous Galerkin approximation](https://mlsanity.com/paper/2101.11810/) (2021)
- [A Locally Conservative Mixed Finite Element Framework for Coupled Hydro-Mechanical-Chemical Processes in Heterogeneous Porous Media](https://mlsanity.com/paper/2010.04994/) (2020)
- [Benchmarks for single-phase flow in fractured porous media](https://mlsanity.com/paper/1701.01496/) (2017)

## Built on by (most cited first, 12 of 12)

- [gLaSDI: Parametric Physics-informed Greedy Latent Space Dynamics Identification](https://mlsanity.com/paper/2204.12005/) (2022)
- [Physics-informed active learning with simultaneous weak-form latent space dynamics identification](https://mlsanity.com/paper/2407.00337/) (2024)
- [Generative Adversarial Reduced Order Modelling](https://mlsanity.com/paper/2305.15881/) (2023)
- [State Representation Learning Using an Unbalanced Atlas](https://mlsanity.com/paper/2305.10267/) (2023)
- [Surrogates for Physics-based and Data-driven Modelling of Parametric Systems: Review and New Perspectives](https://mlsanity.com/paper/2603.12870/) (2026)
- [Enhancing Dynamic Mode Decomposition Workflow with In-Situ Visualization and Data Compression](https://mlsanity.com/paper/2208.07767/) (2022)
- [Contrastive Learning Boosts Deterministic and Generative Models for Weather Data](https://mlsanity.com/paper/2603.24744/) (2026)
- [Progressive reduced order modeling: empowering data-driven modeling with selective knowledge transfer](https://mlsanity.com/paper/2310.03770/) (2023)
- [Efficient machine-learning surrogates for large-scale geological carbon and energy storage](https://mlsanity.com/paper/2310.07461/) (2023)
- [Maximum Manifold Capacity Representations in State Representation Learning](https://mlsanity.com/paper/2405.13848/) (2024)
- [Physics-Informed Neural Networks for Control of Single-Phase Flow Systems Governed by Partial Differential Equations](https://mlsanity.com/paper/2506.06188/) (2025)
- [Sparse POD Mode Selection and Manifold Dimensionality Reduction with Neural Networks](https://mlsanity.com/paper/2605.27756/) (2026)

## Abstract (arXiv)

We propose a unified data-driven reduced order model (ROM) that bridges the performance gap between linear and nonlinear manifold approaches. Deep learning ROM (DL-ROM) using deep-convolutional autoencoders (DC-AE) has been shown to capture nonlinear solution manifolds but fails to perform adequately when linear subspace approaches such as proper orthogonal decomposition (POD) would be optimal. Besides, most DL-ROM models rely on convolutional layers, which might limit its application to only a structured mesh. The proposed framework in this study relies on the combination of an autoencoder (AE) and Barlow Twins (BT) self-supervised learning, where BT maximizes the information content of the embedding with the latent space through a joint embedding architecture. Through a series of benchmark problems of natural convection in porous media, BT-AE performs better than the previous DL-ROM framework by providing comparable results to POD-based approaches for problems where the solution lies within a linear subspace as well as DL-ROM autoencoder-based techniques where the solution lies on a nonlinear manifold; consequently, bridges the gap between linear and nonlinear reduced manifolds. Furthermore, this BT-AE framework can operate on unstructured meshes, which provides flexibility in its application to standard numerical solvers, on-site measurements, experimental data, or a combination of these sources.