# Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles

arXiv: https://arxiv.org/abs/2302.09748 · HTML: https://mlsanity.com/paper/2302.09748/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 86 references that are in the map.
- First posted on arXiv 20 Feb 2023.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons](https://mlsanity.com/paper/2201.07766/) (2022)
- [Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks](https://mlsanity.com/paper/1802.07486/) (2018)
- [Dynamic mode decomposition with control](https://mlsanity.com/paper/1409.6358/) (2014)
- [Transformers for Modeling Physical Systems](https://mlsanity.com/paper/2010.03957/) (2020)
- [A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems](https://mlsanity.com/paper/1908.05823/) (2019)
- [Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systems](https://mlsanity.com/paper/2203.15706/) (2022)
- [Deep learning observables in computational fluid dynamics](https://mlsanity.com/paper/1903.03040/) (2019)
- [Solving Bayesian Inverse Problems via Variational Autoencoders](https://mlsanity.com/paper/1912.04212/) (2019)
- [AutoDEUQ: Automated Deep Ensemble with Uncertainty Quantification](https://mlsanity.com/paper/2110.13511/) (2021)
- [Data-driven super-parameterization using deep learning: Experimentation with multi-scale Lorenz 96 systems and transfer-learning](https://mlsanity.com/paper/2002.11167/) (2020)
- [Enhanced data efficiency using deep neural networks and Gaussian processes for aerodynamic design optimization](https://mlsanity.com/paper/2008.06731/) (2020)
- [Assessments of epistemic uncertainty using Gaussian stochastic weight averaging for fluid-flow regression](https://mlsanity.com/paper/2109.08248/) (2021)
- [Latent-space time evolution of non-intrusive reduced-order models using Gaussian process emulation](https://mlsanity.com/paper/2007.12167/) (2020)
- [AgEBO-Tabular: Joint Neural Architecture and Hyperparameter Search with Autotuned Data-Parallel Training for Tabular Data](https://mlsanity.com/paper/2010.16358/) (2020)
- [Accelerating PDE-constrained Inverse Solutions with Deep Learning and Reduced Order Models](https://mlsanity.com/paper/1912.08864/) (2019)
- [An efficient, globally convergent method for optimization under uncertainty using adaptive model reduction and sparse grids](https://mlsanity.com/paper/1811.00177/) (2018)
- [Data-driven nonintrusive reduced order modeling for dynamical systems with moving boundaries using Gaussian process regression](https://mlsanity.com/paper/2103.09790/) (2021)

## Built on by (most cited first, 5 of 5)

- [Multimodal Atmospheric Super-Resolution With Deep Generative Models](https://mlsanity.com/paper/2506.22780/) (2025)
- [Towards Reliable Uncertainty Quantification via Deep Ensembles in Multi-output Regression Task](https://mlsanity.com/paper/2303.16210/) (2023)
- [Leveraging Interpolation Models and Error Bounds for Verifiable Scientific Machine Learning](https://mlsanity.com/paper/2404.03586/) (2024)
- [Empowering Bayesian Neural Networks with Functional Priors through Anchored Ensembling for Mechanics Surrogate Modeling Applications](https://mlsanity.com/paper/2409.05234/) (2024)
- [Higher order quantum reservoir computing for non-intrusive reduced-order models](https://mlsanity.com/paper/2407.21602/) (2024)

## Abstract (arXiv)

Classical problems in computational physics such as data-driven forecasting and signal reconstruction from sparse sensors have recently seen an explosion in deep neural network (DNN) based algorithmic approaches. However, most DNN models do not provide uncertainty estimates, which are crucial for establishing the trustworthiness of these techniques in downstream decision making tasks and scenarios. In recent years, ensemble-based methods have achieved significant success for the uncertainty quantification in DNNs on a number of benchmark problems. However, their performance on real-world applications remains under-explored. In this work, we present an automated approach to DNN discovery and demonstrate how this may also be utilized for ensemble-based uncertainty quantification. Specifically, we propose the use of a scalable neural and hyperparameter architecture search for discovering an ensemble of DNN models for complex dynamical systems. We highlight how the proposed method not only discovers high-performing neural network ensembles for our tasks, but also quantifies uncertainty seamlessly. This is achieved by using genetic algorithms and Bayesian optimization for sampling the search space of neural network architectures and hyperparameters. Subsequently, a model selection approach is used to identify candidate models for an ensemble set construction. Afterwards, a variance decomposition approach is used to estimate the uncertainty of the predictions from the ensemble. We demonstrate the feasibility of this framework for two tasks - forecasting from historical data and flow reconstruction from sparse sensors for the sea-surface temperature. We demonstrate superior performance from the ensemble in contrast with individual high-performing models and other benchmarks.