# Repulsive Ensembles for Bayesian Inference in Physics-informed Neural Networks

arXiv: https://arxiv.org/abs/2505.17308 · HTML: https://mlsanity.com/paper/2505.17308/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 18 of 18)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [On Linear Identifiability of Learned Representations](https://mlsanity.com/paper/2007.00810/) (2020)
- [Repulsive Deep Ensembles are Bayesian](https://mlsanity.com/paper/2106.11642/) (2021)
- [Function Space Particle Optimization for Bayesian Neural Networks](https://mlsanity.com/paper/1902.09754/) (2019)
- [Improved Training of Physics-Informed Neural Networks with Model Ensembles](https://mlsanity.com/paper/2204.05108/) (2022)
- [Multi-Output Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Uncertainties](https://mlsanity.com/paper/2202.01710/) (2022)
- [Learning and discovering multiple solutions using physics-informed neural networks with random initialization and deep ensemble](https://mlsanity.com/paper/2503.06320/) (2025)
- [Ensemble learning for Physics Informed Neural Networks: a Gradient Boosting approach](https://mlsanity.com/paper/2302.13143/) (2023)
- [Physics-informed Neural Networks with Unknown Measurement Noise](https://mlsanity.com/paper/2211.15498/) (2022)
- [Input-gradient space particle inference for neural network ensembles](https://mlsanity.com/paper/2306.02775/) (2023)

## Built on by (most cited first, 1 of 1)

- [Uncertainty Quantification in PINNs for Turbulent Flows: Bayesian Inference and Repulsive Ensembles](https://mlsanity.com/paper/2604.17156/) (2026)

## Abstract (arXiv)

Physics-informed neural networks (PINNs) have proven an effective tool for solving differential equations, in particular when considering non-standard or ill-posed settings. When inferring solutions and parameters of the differential equation from data, uncertainty estimates are preferable to point estimates, as they give an idea about the accuracy of the solution. In this work, we consider the inverse problem and employ repulsive ensembles of PINNs (RE-PINN) for obtaining such estimates. The repulsion is implemented by adding a particular repulsive term to the loss function, which has the property that the ensemble predictions correspond to the true Bayesian posterior in the limit of infinite ensemble members. Where possible, we compare the ensemble predictions to Monte Carlo baselines. Whereas the standard ensemble tends to collapse to maximum-a-posteriori solutions, the repulsive ensemble produces significantly more accurate uncertainty estimates and exhibits higher sample diversity.