# A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems

arXiv: https://arxiv.org/abs/2202.04648 · HTML: https://mlsanity.com/paper/2202.04648/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 166 references that are in the map.
- First posted on arXiv 9 Feb 2022.

## Builds on (most cited first, 29 of 29)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Wasserstein Auto-Encoders](https://mlsanity.com/paper/1711.01558/) (2017)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/1801.06879/) (2018)
- [Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems](https://mlsanity.com/paper/1809.08327/) (2018)
- [Adversarial Uncertainty Quantification in Physics-Informed Neural Networks](https://mlsanity.com/paper/1811.04026/) (2018)
- [A physics-informed variational DeepONet for predicting the crack path in brittle materials](https://mlsanity.com/paper/2108.06905/) (2021)
- [Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification](https://mlsanity.com/paper/1802.00850/) (2018)
- [Manifold Gaussian Processes for Regression](https://mlsanity.com/paper/1402.5876/) (2014)
- [Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations](https://mlsanity.com/paper/1703.10230/) (2017)
- [A survey of dimensionality reduction techniques](https://mlsanity.com/paper/1403.2877/) (2014)
- [Sparse Polynomial Chaos Expansions: Literature Survey and Benchmark](https://mlsanity.com/paper/2002.01290/) (2020)
- [Normalizing field flows: Solving forward and inverse stochastic differential equations using physics-informed flow models](https://mlsanity.com/paper/2108.12956/) (2021)
- [Random-projection ensemble classification](https://mlsanity.com/paper/1504.04595/) (2015)
- [Data-driven polynomial chaos expansion for machine learning regression](https://mlsanity.com/paper/1808.03216/) (2018)
- [On the Latent Space of Wasserstein Auto-Encoders](https://mlsanity.com/paper/1802.03761/) (2018)
- [Extending classical surrogate modelling to high-dimensions through supervised dimensionality reduction: a data-driven approach](https://mlsanity.com/paper/1812.06309/) (2018)
- [Manifold learning-based polynomial chaos expansions for high-dimensional surrogate models](https://mlsanity.com/paper/2107.09814/) (2021)
- [Wasserstein Generative Adversarial Uncertainty Quantification in Physics-Informed Neural Networks](https://mlsanity.com/paper/2108.13054/) (2021)
- [Compressive sensing adaptation for polynomial chaos expansions](https://mlsanity.com/paper/1801.01961/) (2018)
- [Gradient-based Optimization for Regression in the Functional Tensor-Train Format](https://mlsanity.com/paper/1801.00885/) (2018)
- [Grassmannian diffusion maps based dimension reduction and classification for high-dimensional data](https://mlsanity.com/paper/2009.07547/) (2020)
- [Tensor Random Projection for Low Memory Dimension Reduction](https://mlsanity.com/paper/2105.00105/) (2021)
- [Grassmannian diffusion maps based surrogate modeling via geometric harmonics](https://mlsanity.com/paper/2109.13805/) (2021)
- [Learning in High-Dimensional Multimedia Data: The State of the Art](https://mlsanity.com/paper/1707.02683/) (2017)
- [Robust Adaptive Least Squares Polynomial Chaos Expansions in High-Frequency Applications](https://mlsanity.com/paper/1912.07725/) (2019)
- [Reduced Wiener Chaos representation of random fields via basis adaptation and projection](https://mlsanity.com/paper/1603.04803/) (2016)

## Built on by (most cited first, 20 of 20)

- [On the influence of over-parameterization in manifold based surrogates and deep neural operators](https://mlsanity.com/paper/2203.05071/) (2022)
- [L-HYDRA: Multi-Head Physics-Informed Neural Networks](https://mlsanity.com/paper/2301.02152/) (2023)
- [Neural Chaos: A Spectral Stochastic Neural Operator](https://mlsanity.com/paper/2502.11835/) (2025)
- [Learning in latent spaces improves the predictive accuracy of deep neural operators](https://mlsanity.com/paper/2304.07599/) (2023)
- [End-to-end Conditional Robust Optimization](https://mlsanity.com/paper/2403.04670/) (2024)
- [Polynomial Chaos Expansion for Operator Learning](https://mlsanity.com/paper/2508.20886/) (2025)
- [Polynomial Chaos Expansions on Principal Geodesic Grassmannian Submanifolds for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/2401.16683/) (2024)
- [PCENet: High Dimensional Surrogate Modeling for Learning Uncertainty](https://mlsanity.com/paper/2202.05063/) (2022)
- [E-PINNs: Epistemic Physics-Informed Neural Networks](https://mlsanity.com/paper/2503.19333/) (2025)
- [Probabilistic partition of unity networks for high-dimensional regression problems](https://mlsanity.com/paper/2210.02694/) (2022)
- [Dimensionality reduction can be used as a surrogate model for high-dimensional forward uncertainty quantification](https://mlsanity.com/paper/2402.04582/) (2024)
- [Neural active manifolds: nonlinear dimensionality reduction for uncertainty quantification](https://mlsanity.com/paper/2408.03534/) (2024)
- [Stratospheric aerosol source inversion: Noise, variability, and uncertainty quantification](https://mlsanity.com/paper/2409.06846/) (2024)
- [Learning signals defined on graphs with optimal transport and Gaussian process regression](https://mlsanity.com/paper/2410.15721/) (2024)
- [Solving Coupled Differential Equation Groups Using PINO-CDE](https://mlsanity.com/paper/2210.00222/) (2022)
- [Model-Agnostic Interpretation Framework in Machine Learning: A Comparative Study in NBA Sports](https://mlsanity.com/paper/2401.02630/) (2024)
- [A Resolution Independent Neural Operator](https://mlsanity.com/paper/2407.13010/) (2024)
- [Visualizing definitional divergence in high-dimensional data by manifold alignment: Application to 3D right ventricular strain computations](https://mlsanity.com/paper/2501.12178/) (2025)
- [Conformalized Polynomial Chaos Expansion for Uncertainty-aware Surrogate Modeling](https://mlsanity.com/paper/2510.22375/) (2025)
- [Structured Neural Chaos: An Adaptive Surrogate Modeling Framework for Functional Uncertainty Quantification and Global Sensitivity Analysis](https://mlsanity.com/paper/2607.28903/) (2026)

## Abstract (arXiv)

Constructing surrogate models for uncertainty quantification (UQ) on complex partial differential equations (PDEs) having inherently high-dimensional O(10≥ 2) stochastic inputs (e.g., forcing terms, boundary conditions, initial conditions) poses tremendous challenges. The curse of dimensionality can be addressed with suitable unsupervised learning techniques used as a pre-processing tool to encode inputs onto lower-dimensional subspaces while retaining its structural information and meaningful properties. In this work, we review and investigate thirteen dimension reduction methods including linear and nonlinear, spectral, blind source separation, convex and non-convex methods and utilize the resulting embeddings to construct a mapping to quantities of interest via polynomial chaos expansions (PCE). We refer to the general proposed approach as manifold PCE (m-PCE), where manifold corresponds to the latent space resulting from any of the studied dimension reduction methods. To investigate the capabilities and limitations of these methods we conduct numerical tests for three physics-based systems (treated as black-boxes) having high-dimensional stochastic inputs of varying complexity modeled as both Gaussian and non-Gaussian random fields to investigate the effect of the intrinsic dimensionality of input data. We demonstrate both the advantages and limitations of the unsupervised learning methods and we conclude that a suitable m-PCE model provides a cost-effective approach compared to alternative algorithms proposed in the literature, including recently proposed expensive deep neural network-based surrogates and can be readily applied for high-dimensional UQ in stochastic PDEs.