# Distilling Machine Learning's Added Value: Pareto Fronts in Atmospheric Applications

arXiv: https://arxiv.org/abs/2408.02161 · HTML: https://mlsanity.com/paper/2408.02161/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 89 references that are in the map.
- First posted on arXiv 4 Aug 2024.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation](https://mlsanity.com/paper/1912.05074/) (2019)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [WeatherBench: A benchmark dataset for data-driven weather forecasting](https://mlsanity.com/paper/2002.00469/) (2020)
- [GraphCast: Learning skillful medium-range global weather forecasting](https://mlsanity.com/paper/2212.12794/) (2022)
- [Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey](https://mlsanity.com/paper/2006.11371/) (2020)
- [Pareto Multi-Task Learning](https://mlsanity.com/paper/1912.12854/) (2019)
- [Contemporary Symbolic Regression Methods and their Relative Performance](https://mlsanity.com/paper/2107.14351/) (2021)
- [Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast](https://mlsanity.com/paper/2211.02556/) (2022)
- [Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey](https://mlsanity.com/paper/1911.12116/) (2019)
- [Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geoscience](https://mlsanity.com/paper/2202.03407/) (2022)
- [Climbing down Charney's ladder: Machine Learning and the post-Dennard era of computational climate science](https://mlsanity.com/paper/2005.11862/) (2020)
- [Combining distribution-based neural networks to predict weather forecast probabilities](https://mlsanity.com/paper/2103.14430/) (2021)
- [Sherpa: Robust Hyperparameter Optimization for Machine Learning](https://mlsanity.com/paper/2005.04048/) (2020)

## Built on by (most cited first, 3 of 3)

- [Crowdsourcing the Frontier: Advancing Hybrid Physics-ML Climate Simulation via a 50,000 Kaggle Competition](https://mlsanity.com/paper/2511.20963/) (2025)
- [How Can Machine Learning Emulators Best Support Climate Science?](https://mlsanity.com/paper/2603.22320/) (2026)
- [Learning Prognostic Variables for AI Convective Parameterizations via Symbolic Distillation](https://mlsanity.com/paper/2609.24882/) (2026)

## Abstract (arXiv)

The added value of machine learning for weather and climate applications is measurable through performance metrics, but explaining it remains challenging, particularly for large deep learning models. Inspired by climate model hierarchies, we propose that a full hierarchy of Pareto-optimal models, defined within an appropriately determined error-complexity plane, can guide model development and help understand the models' added value. We demonstrate the use of Pareto fronts in atmospheric physics through three sample applications, with hierarchies ranging from semi-empirical models with minimal parameters to deep learning algorithms. First, in cloud cover parameterization, we find that neural networks identify nonlinear relationships between cloud cover and its thermodynamic environment, and assimilate previously neglected features such as vertical gradients in relative humidity that improve the representation of low cloud cover. This added value is condensed into a ten-parameter equation that rivals deep learning models. Second, we establish a machine learning model hierarchy for emulating shortwave radiative transfer, distilling the importance of bidirectional vertical connectivity for accurately representing absorption and scattering, especially for multiple cloud layers. Third, we emphasize the importance of convective organization information when modeling the relationship between tropical precipitation and its surrounding environment. We discuss the added value of temporal memory when high-resolution spatial information is unavailable, with implications for precipitation parameterization. Therefore, by comparing data-driven models directly with existing schemes using Pareto optimality, we promote process understanding by hierarchically unveiling system complexity, with the hope of improving the trustworthiness of machine learning models in atmospheric applications.