# Output-weighted and relative entropy loss functions for deep learning precursors of extreme events

arXiv: https://arxiv.org/abs/2112.00825 · HTML: https://mlsanity.com/paper/2112.00825/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 58 references that are in the map.
- First posted on arXiv 1 Dec 2021.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Machine Learning for Fluid Mechanics](https://mlsanity.com/paper/1905.11075/) (2019)
- [Deep learning to represent sub-grid processes in climate models](https://mlsanity.com/paper/1806.04731/) (2018)
- [Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems](https://mlsanity.com/paper/1912.08177/) (2019)
- [Will Artificial Intelligence supersede Earth System and Climate Models?](https://mlsanity.com/paper/2101.09126/) (2021)
- [Neural Closure Models for Dynamical Systems](https://mlsanity.com/paper/2012.13869/) (2020)
- [A sequential sampling strategy for extreme event statistics in nonlinear dynamical systems](https://mlsanity.com/paper/1804.07240/) (2018)
- [Output-weighted optimal sampling for Bayesian regression and rare event statistics using few samples](https://mlsanity.com/paper/1907.07552/) (2019)
- [Are extreme dissipation events predictable in turbulent fluid flows?](https://mlsanity.com/paper/1807.10263/) (2018)
- [An adjoint-based approach for finding invariant solutions of Navier-Stokes equations](https://mlsanity.com/paper/1508.06363/) (2015)
- [Model-assisted deep learning of rare extreme events from partial observations](https://mlsanity.com/paper/2111.04857/) (2021)

## Built on by (most cited first, 8 of 8)

- [A Multi-Scale Deep Learning Framework for Projecting Weather Extremes](https://mlsanity.com/paper/2210.12137/) (2022)
- [Entropy-based Guidance of Deep Neural Networks for Accelerated Convergence and Improved Performance](https://mlsanity.com/paper/2308.14938/) (2023)
- [Using Explainable AI and Transfer Learning to understand and predict the maintenance of Atlantic blocking with limited observational data](https://mlsanity.com/paper/2404.08613/) (2024)
- [Dynamics-Informed Deep Learning for Predicting Extreme Events](https://mlsanity.com/paper/2603.10777/) (2026)
- [A generalized likelihood-weighted optimal sampling algorithm for rare-event probability quantification](https://mlsanity.com/paper/2310.14457/) (2023)
- [Learning Response-Statistic Shifts and Parametric Roll Episodes from Wave--Vessel Time Series via LSTM Functional Models](https://mlsanity.com/paper/2603.24431/) (2026)
- [A numerical study into neural network surrogate model performance for uncertainty propagation](https://mlsanity.com/paper/2605.16078/) (2026)
- [Uncovering Extreme Event Mechanisms for Prediction and Control with Sensitivity-Balanced Projections](https://mlsanity.com/paper/2606.05618/) (2026)

## Abstract (arXiv)

Many scientific and engineering problems require accurate models of dynamical systems with rare and extreme events. Such problems present a challenging task for data-driven modelling, with many naive machine learning methods failing to predict or accurately quantify such events. One cause for this difficulty is that systems with extreme events, by definition, yield imbalanced datasets and that standard loss functions easily ignore rare events. That is, metrics for goodness of fit used to train models are not designed to ensure accuracy on rare events. This work seeks to improve the performance of regression models for extreme events by considering loss functions designed to highlight outliers. We propose a novel loss function, the adjusted output weighted loss, and extend the applicability of relative entropy based loss functions to systems with low dimensional output. The proposed functions are tested using several cases of dynamical systems exhibiting extreme events and shown to significantly improve accuracy in predictions of extreme events.