# Navigating the Noise: Bringing Clarity to ML Parameterization Design with O(100) Ensembles

arXiv: https://arxiv.org/abs/2309.16177 · HTML: https://mlsanity.com/paper/2309.16177/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 51 references that are in the map.
- First posted on arXiv 28 Sep 2023.

## Builds on (most cited first, 8 of 8)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Deep learning to represent sub-grid processes in climate models](https://mlsanity.com/paper/1806.04731/) (2018)
- [Quasi-hyperbolic momentum and Adam for deep learning](https://mlsanity.com/paper/1810.06801/) (2018)
- [Machine Learning for Stochastic Parameterization: Generative Adversarial Networks in the Lorenz '96 Model](https://mlsanity.com/paper/1909.04711/) (2019)
- [Climate-Invariant Machine Learning](https://mlsanity.com/paper/2112.08440/) (2021)
- [ClimART: A Benchmark Dataset for Emulating Atmospheric Radiative Transfer in Weather and Climate Models](https://mlsanity.com/paper/2111.14671/) (2021)
- [A Fortran-Keras Deep Learning Bridge for Scientific Computing](https://mlsanity.com/paper/2004.10652/) (2020)
- [Multi-fidelity climate model parameterization for better generalization and extrapolation](https://mlsanity.com/paper/2309.10231/) (2023)

## Built on by (most cited first, 3 of 3)

- [Crowdsourcing the Frontier: Advancing Hybrid Physics-ML Climate Simulation via a 50,000 Kaggle Competition](https://mlsanity.com/paper/2511.20963/) (2025)
- [Decomposing Ensemble Spread in Lorenz '96 With Learned Stochastic Parameterizations](https://mlsanity.com/paper/2605.22242/) (2026)
- [No Epoch Like the Present: Robust Climate Emulation Requires Out-of-Distribution Generalisation](https://mlsanity.com/paper/2605.22248/) (2026)

## Abstract (arXiv)

Machine-learning (ML) parameterizations of subgrid processes (here of turbulence, convection, and radiation) may one day replace conventional parameterizations by emulating high-resolution physics without the cost of explicit simulation. However, uncertainty about the relationship between offline and online performance (i.e., when integrated with a large-scale general circulation model (GCM)) hinders their development. Much of this uncertainty stems from limited sampling of the noisy, emergent effects of upstream ML design decisions on downstream online hybrid simulation. Our work rectifies the sampling issue via the construction of a semi-automated, end-to-end pipeline for O(100) size ensembles of hybrid simulations, revealing important nuances in how systematic reductions in offline error manifest in changes to online error and online stability. For example, removing dropout and switching from a Mean Squared Error (MSE) to a Mean Absolute Error (MAE) loss both reduce offline error, but they have opposite effects on online error and online stability. Other design decisions, like incorporating memory, converting moisture input from specific humidity to relative humidity, using batch normalization, and training on multiple climates do not come with any such compromises. Finally, we show that ensemble sizes of O(100) may be necessary to reliably detect causally relevant differences online. By enabling rapid online experimentation at scale, we can empirically settle debates regarding subgrid ML parameterization design that would have otherwise remained unresolved in the noise.