# CIRA Guide to Custom Loss Functions for Neural Networks in Environmental Sciences -- Version 1

arXiv: https://arxiv.org/abs/2106.09757 · HTML: https://mlsanity.com/paper/2106.09757/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 44 references that are in the map.
- First posted on arXiv 17 Jun 2021.

## Builds on (most cited first, 15 of 15)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017)
- [Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data](https://mlsanity.com/paper/1612.08544/) (2016)
- [A Review on Deep Learning Techniques Applied to Semantic Segmentation](https://mlsanity.com/paper/1704.06857/) (2017)
- [Anticipating Traffic Accidents with Adaptive Loss and Large-scale Incident DB](https://mlsanity.com/paper/1804.02675/) (2018)
- [An Alternative Probabilistic Interpretation of the Huber Loss](https://mlsanity.com/paper/1911.02088/) (2019)
- [Development and Interpretation of a Neural Network-Based Synthetic Radar Reflectivity Estimator Using GOES-R Satellite Observations](https://mlsanity.com/paper/2004.07906/) (2020)
- [CoPhy-PGNN: Learning Physics-guided Neural Networks with Competing Loss Functions for Solving Eigenvalue Problems](https://mlsanity.com/paper/2007.01420/) (2020)
- [Tropical and Extratropical Cyclone Detection Using Deep Learning](https://mlsanity.com/paper/2005.09056/) (2020)
- [Controlled abstention neural networks for identifying skillful predictions for regression problems](https://mlsanity.com/paper/2104.08236/) (2021)
- [Controlled abstention neural networks for identifying skillful predictions for classification problems](https://mlsanity.com/paper/2104.08281/) (2021)
- [Adaptive Loss Function for Super Resolution Neural Networks Using Convex Optimization Techniques](https://mlsanity.com/paper/2001.07766/) (2020)

## Built on by (most cited first, 8 of 8)

- [Increasing the accuracy and resolution of precipitation forecasts using deep generative models](https://mlsanity.com/paper/2203.12297/) (2022)
- [Kilometer-Scale Convection Allowing Model Emulation using Generative Diffusion Modeling](https://mlsanity.com/paper/2408.10958/) (2024)
- [Advancing Parsimonious Deep Learning Weather Prediction using the HEALPix Mesh](https://mlsanity.com/paper/2311.06253/) (2023)
- [Can we integrate spatial verification methods into neural-network loss functions for atmospheric science?](https://mlsanity.com/paper/2203.11141/) (2022)
- [Physics-constrained deep learning postprocessing of temperature and humidity](https://mlsanity.com/paper/2212.04487/) (2022)
- [Machine Learning Estimation of Maximum Vertical Velocity from Radar](https://mlsanity.com/paper/2310.09392/) (2023)
- [Custom Loss Functions in Fuel Moisture Modeling](https://mlsanity.com/paper/2501.10401/) (2025)
- [Data-Driven Forward and Inverse Modeling of V-Beam Thermal Sensors](https://mlsanity.com/paper/2607.09752/) (2026)

## Abstract (arXiv)

Neural networks are increasingly used in environmental science applications. Furthermore, neural network models are trained by minimizing a loss function, and it is crucial to choose the loss function very carefully for environmental science applications, as it determines what exactly is being optimized. Standard loss functions do not cover all the needs of the environmental sciences, which makes it important for scientists to be able to develop their own custom loss functions so that they can implement many of the classic performance measures already developed in environmental science, including measures developed for spatial model verification. However, there are very few resources available that cover the basics of custom loss function development comprehensively, and to the best of our knowledge none that focus on the needs of environmental scientists. This document seeks to fill this gap by providing a guide on how to write custom loss functions targeted toward environmental science applications. Topics include the basics of writing custom loss functions, common pitfalls, functions to use in loss functions, examples such as fractions skill score as loss function, how to incorporate physical constraints, discrete and soft discretization, and concepts such as focal, robust, and adaptive loss. While examples are currently provided in this guide for Python with Keras and the TensorFlow backend, the basic concepts also apply to other environments, such as Python with PyTorch. Similarly, while the sample loss functions provided here are from meteorology, these are just examples of how to create custom loss functions. Other fields in the environmental sciences have very similar needs for custom loss functions, e.g., for evaluating spatial forecasts effectively, and the concepts discussed here can be applied there as well. All code samples are provided in a GitHub repository.