# Explainable Artificial Intelligence for Bayesian Neural Networks: Towards trustworthy predictions of ocean dynamics

arXiv: https://arxiv.org/abs/2205.00202 · HTML: https://mlsanity.com/paper/2205.00202/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 73 references that are in the map.
- First posted on arXiv 30 Apr 2022.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [Tackling Climate Change with Machine Learning](https://mlsanity.com/paper/1906.05433/) (2019)
- [Explaining individual predictions when features are dependent: More accurate approximations to Shapley values](https://mlsanity.com/paper/1903.10464/) (2019)
- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers](https://mlsanity.com/paper/1604.00825/) (2016)
- [The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01602/) (2022)
- [Practical Deep Learning with Bayesian Principles](https://mlsanity.com/paper/1906.02506/) (2019)
- [Reliable Post hoc Explanations: Modeling Uncertainty in Explainability](https://mlsanity.com/paper/2008.05030/) (2020)
- [TensorFlow Distributions](https://mlsanity.com/paper/1711.10604/) (2017)
- ["What is Relevant in a Text Document?": An Interpretable Machine Learning Approach](https://mlsanity.com/paper/1612.07843/) (2016)
- [Rethinking Explainability as a Dialogue: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01875/) (2022)
- [Physically Interpretable Neural Networks for the Geosciences: Applications to Earth System Variability](https://mlsanity.com/paper/1912.01752/) (2019)
- [Being Bayesian about Categorical Probability](https://mlsanity.com/paper/2002.07965/) (2020)
- [Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geoscience](https://mlsanity.com/paper/2202.03407/) (2022)
- [How Much Can I Trust You? -- Quantifying Uncertainties in Explaining Neural Networks](https://mlsanity.com/paper/2006.09000/) (2020)
- [Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset](https://mlsanity.com/paper/2103.10005/) (2021)
- [Combining distribution-based neural networks to predict weather forecast probabilities](https://mlsanity.com/paper/2103.14430/) (2021)
- [On the Validity of Bayesian Neural Networks for Uncertainty Estimation](https://mlsanity.com/paper/1912.01530/) (2019)
- [Prior choice affects ability of Bayesian neural networks to identify unknowns](https://mlsanity.com/paper/2005.04987/) (2020)
- [Objective discovery of dominant dynamical processes with intelligible machine learning](https://mlsanity.com/paper/2106.12963/) (2021)

## Built on by (most cited first, 11 of 11)

- [Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Science](https://mlsanity.com/paper/2303.00652/) (2023)
- [Learning Closed-form Equations for Subgrid-scale Closures from High-fidelity Data: Promises and Challenges](https://mlsanity.com/paper/2306.05014/) (2023)
- [Southern Ocean Dynamics Under Climate Change: New Knowledge Through Physics-Guided Machine Learning](https://mlsanity.com/paper/2310.13916/) (2023)
- [On the choice of training data for machine learning of geostrophic mesoscale turbulence](https://mlsanity.com/paper/2307.00734/) (2023)
- [Characterizing climate pathways using feature importance on echo state networks](https://mlsanity.com/paper/2310.08495/) (2023)
- [Bayesian neural networks for the probabilistic forecasting of wind direction and speed using ocean data](https://mlsanity.com/paper/2206.08953/) (2022)
- [The Importance of Architecture Choice in Deep Learning for Climate Applications](https://mlsanity.com/paper/2402.13979/) (2024)
- [IMAFD: An Interpretable Multi-stage Approach to Flood Detection from time series Multispectral Data](https://mlsanity.com/paper/2405.07916/) (2024)
- [A Unified Framework for Uncertainty-Aware Explainable Artificial Intelligence: A Case Study in Power Quality Disturbance Classification](https://mlsanity.com/paper/2605.21114/) (2026)
- [A Hybrid LSTM--Vision Transformer Architecture for Predicting HRRR Forecast Errors](https://mlsanity.com/paper/2606.19026/) (2026)
- [From objective discovery to prediction of global ocean eco-provinces: A pathway for trustworthy learning](https://mlsanity.com/paper/2609.13206/) (2026)

## Abstract (arXiv)

The trustworthiness of neural networks is often challenged because they lack the ability to express uncertainty and explain their skill. This can be problematic given the increasing use of neural networks in high stakes decision-making such as in climate change applications. We address both issues by successfully implementing a Bayesian Neural Network (BNN), where parameters are distributions rather than deterministic, and applying novel implementations of explainable AI (XAI) techniques. The uncertainty analysis from the BNN provides a comprehensive overview of the prediction more suited to practitioners' needs than predictions from a classical neural network. Using a BNN means we can calculate the entropy (i.e. uncertainty) of the predictions and determine if the probability of an outcome is statistically significant. To enhance trustworthiness, we also spatially apply the two XAI techniques of Layer-wise Relevance Propagation (LRP) and SHapley Additive exPlanation (SHAP) values. These XAI methods reveal the extent to which the BNN is suitable and/or trustworthy. Using two techniques gives a more holistic view of BNN skill and its uncertainty, as LRP considers neural network parameters, whereas SHAP considers changes to outputs. We verify these techniques using comparison with intuition from physical theory. The differences in explanation identify potential areas where new physical theory guided studies are needed.