# The Importance of Architecture Choice in Deep Learning for Climate Applications

arXiv: https://arxiv.org/abs/2402.13979 · HTML: https://mlsanity.com/paper/2402.13979/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 51 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Builds on (most cited first, 15 of 15)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [When Explanations Lie: Why Many Modified BP Attributions Fail](https://mlsanity.com/paper/1912.09818/) (2019)
- [Will Artificial Intelligence supersede Earth System and Climate Models?](https://mlsanity.com/paper/2101.09126/) (2021)
- [Bridging observation, theory and numerical simulation of the ocean using Machine Learning](https://mlsanity.com/paper/2104.12506/) (2021)
- [Explainable Artificial Intelligence for Bayesian Neural Networks: Towards trustworthy predictions of ocean dynamics](https://mlsanity.com/paper/2205.00202/) (2022)
- [Interpretable and Robust AI in EEG Systems: A Survey](https://mlsanity.com/paper/2304.10755/) (2023)
- [Bayesian neural networks for weak solution of PDEs with uncertainty quantification](https://mlsanity.com/paper/2101.04879/) (2021)
- [Southern Ocean Dynamics Under Climate Change: New Knowledge Through Physics-Guided Machine Learning](https://mlsanity.com/paper/2310.13916/) (2023)
- [Bayesian neural networks for the probabilistic forecasting of wind direction and speed using ocean data](https://mlsanity.com/paper/2206.08953/) (2022)

## Abstract (arXiv)

Machine Learning has become a pervasive tool in climate science applications. However, current models fail to address nonstationarity induced by anthropogenic alterations in greenhouse emissions and do not routinely quantify the uncertainty of proposed projections. In this paper, we model the Atlantic Meridional Overturning Circulation (AMOC) which is of major importance to climate in Europe and the US East Coast by transporting warm water to these regions, and has the potential for abrupt collapse. We can generate arbitrarily extreme climate scenarios through arbitrary time scales which we then predict using neural networks. Our analysis shows that the AMOC is predictable using neural networks under a diverse set of climate scenarios. Further experiments reveal that MLPs and Deep Ensembles can learn the physics of the AMOC instead of imitating its progression through autocorrelation. With quantified uncertainty, an intriguing pattern of "spikes" before critical points of collapse in the AMOC casts doubt on previous analyses that predicted an AMOC collapse within this century. Our results show that Bayesian Neural Networks perform poorly compared to more dense architectures and care should be taken when applying neural networks to nonstationary scenarios such as climate projections. Further, our results highlight that big NN models might have difficulty in modeling global Earth System dynamics accurately and be successfully applied in nonstationary climate scenarios due to the physics being challenging for neural networks to capture.