# Unified Focal loss: Generalising Dice and cross entropy-based losses to handle class imbalanced medical image segmentation

arXiv: https://arxiv.org/abs/2102.04525 · HTML: https://mlsanity.com/paper/2102.04525/

## Key facts

- Cited by 88 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 75 references that are in the map.
- First posted on arXiv 8 Feb 2021.

## Builds on (most cited first, 30 of 33)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation](https://mlsanity.com/paper/1809.10486/) (2018)
- [Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images](https://mlsanity.com/paper/1808.08114/) (2018)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [Tversky loss function for image segmentation using 3D fully convolutional deep networks](https://mlsanity.com/paper/1706.05721/) (2017)
- [A survey of loss functions for semantic segmentation](https://mlsanity.com/paper/2006.14822/) (2020)
- [Boundary loss for highly unbalanced segmentation](https://mlsanity.com/paper/1812.07032/) (2018)
- [The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challenge](https://mlsanity.com/paper/1912.01054/) (2019)
- [A Novel Focal Tversky loss function with improved Attention U-Net for lesion segmentation](https://mlsanity.com/paper/1810.07842/) (2018)
- [The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes](https://mlsanity.com/paper/1904.00445/) (2019)
- [DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation](https://mlsanity.com/paper/1506.06448/) (2015)
- [MedGAN: Medical Image Translation using GANs](https://mlsanity.com/paper/1806.06397/) (2018)
- [Brain Tumor Segmentation with Deep Neural Networks](https://mlsanity.com/paper/1505.03540/) (2015)
- [Combo Loss: Handling Input and Output Imbalance in Multi-Organ Segmentation](https://mlsanity.com/paper/1805.02798/) (2018)
- [Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion](https://mlsanity.com/paper/2002.09708/) (2020)
- [AnatomyNet: Deep Learning for Fast and Fully Automated Whole-volume Segmentation of Head and Neck Anatomy](https://mlsanity.com/paper/1808.05238/) (2018)
- [3D Segmentation with Exponential Logarithmic Loss for Highly Unbalanced Object Sizes](https://mlsanity.com/paper/1809.00076/) (2018)
- [USE-Net: incorporating Squeeze-and-Excitation blocks into U-Net for prostate zonal segmentation of multi-institutional MRI datasets](https://mlsanity.com/paper/1904.08254/) (2019)
- [Overfitting of neural nets under class imbalance: Analysis and improvements for segmentation](https://mlsanity.com/paper/1907.10982/) (2019)
- [Combining Noise-to-Image and Image-to-Image GANs: Brain MR Image Augmentation for Tumor Detection](https://mlsanity.com/paper/1905.13456/) (2019)
- [Asymmetric Loss Functions and Deep Densely Connected Networks for Highly Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection](https://mlsanity.com/paper/1803.11078/) (2018)
- [Boundary-weighted Domain Adaptive Neural Network for Prostate MR Image Segmentation](https://mlsanity.com/paper/1902.08128/) (2019)
- [Generalised Wasserstein Dice Score for Imbalanced Multi-class Segmentation using Holistic Convolutional Networks](https://mlsanity.com/paper/1707.00478/) (2017)
- [Brain tumor segmentation with self-ensembled, deeply-supervised 3D U-net neural networks: a BraTS 2020 challenge solution](https://mlsanity.com/paper/2011.01045/) (2020)
- [MIScnn: A Framework for Medical Image Segmentation with Convolutional Neural Networks and Deep Learning](https://mlsanity.com/paper/1910.09308/) (2019)

## Built on by (most cited first, 30 of 88)

- [Topologically faithful image segmentation via induced matching of persistence barcodes](https://mlsanity.com/paper/2211.15272/) (2022)
- [Loss Functions in the Era of Semantic Segmentation: A Survey and Outlook](https://mlsanity.com/paper/2312.05391/) (2023)
- [CoNIC Challenge: Pushing the Frontiers of Nuclear Detection, Segmentation, Classification and Counting](https://mlsanity.com/paper/2303.06274/) (2023)
- [Centerline Boundary Dice Loss for Vascular Segmentation](https://mlsanity.com/paper/2407.01517/) (2024)
- [MicroSegNet: A Deep Learning Approach for Prostate Segmentation on Micro-Ultrasound Images](https://mlsanity.com/paper/2305.19956/) (2023)
- [Focus U-Net: A novel dual attention-gated CNN for polyp segmentation during colonoscopy](https://mlsanity.com/paper/2105.07467/) (2021)
- [Calibrating the Dice loss to handle neural network overconfidence for biomedical image segmentation](https://mlsanity.com/paper/2111.00528/) (2021)
- [On the Optimal Combination of Cross-Entropy and Soft Dice Losses for Lesion Segmentation with Out-of-Distribution Robustness](https://mlsanity.com/paper/2209.06078/) (2022)
- [TransRadar: Adaptive-Directional Transformer for Real-Time Multi-View Radar Semantic Segmentation](https://mlsanity.com/paper/2310.02260/) (2023)
- [Ultrasound Nodule Segmentation Using Asymmetric Learning with Simple Clinical Annotation](https://mlsanity.com/paper/2404.14852/) (2024)
- [Joint Learning of Blind Super-Resolution and Crack Segmentation for Realistic Degraded Images](https://mlsanity.com/paper/2302.12491/) (2023)
- [Comparative Analysis of Segment Anything Model and U-Net for Breast Tumor Detection in Ultrasound and Mammography Images](https://mlsanity.com/paper/2306.12510/) (2023)
- [RetiFluidNet: A Self-Adaptive and Multi-Attention Deep Convolutional Network for Retinal OCT Fluid Segmentation](https://mlsanity.com/paper/2209.12468/) (2022)
- [PaveSAM Segment Anything for Pavement Distress](https://mlsanity.com/paper/2409.07295/) (2024)
- [The Dice loss in the context of missing or empty labels: Introducing Phi and ε](https://mlsanity.com/paper/2207.09521/) (2022)
- [Understanding the Tricks of Deep Learning in Medical Image Segmentation: Challenges and Future Directions](https://mlsanity.com/paper/2209.10307/) (2022)
- [Deep-Learning Framework for Optimal Selection of Soil Sampling Sites](https://mlsanity.com/paper/2309.00974/) (2023)
- [Incorporating Boundary Uncertainty into loss functions for biomedical image segmentation](https://mlsanity.com/paper/2111.00533/) (2021)
- [Adaptively Weighted Data Augmentation Consistency Regularization for Robust Optimization under Concept Shift](https://mlsanity.com/paper/2210.01891/) (2022)
- [Mask Focal Loss: A unifying framework for dense crowd counting with canonical object detection networks](https://mlsanity.com/paper/2212.11542/) (2022)
- [General-Purpose Multimodal Transformer meets Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2307.03388/) (2023)
- [An FPGA smart camera implementation of segmentation models for drone wildfire imagery](https://mlsanity.com/paper/2309.01318/) (2023)
- [Deep Multimodal Fusion of Data with Heterogeneous Dimensionality via Projective Networks](https://mlsanity.com/paper/2402.01311/) (2024)
- [Gradient-based Class Weighting for Unsupervised Domain Adaptation in Dense Prediction Visual Tasks](https://mlsanity.com/paper/2407.01327/) (2024)
- [Multi-Scale Spectral Attention Module-based Hyperspectral Segmentation in Autonomous Driving Scenarios](https://mlsanity.com/paper/2506.18682/) (2025)
- [A Deep Learning Approach for Semantic Segmentation of Unbalanced Data in Electron Tomography of Catalytic Materials](https://mlsanity.com/paper/2201.07342/) (2022)
- [Segmentation and Risk Score Prediction of Head and Neck Cancers in PET/CT Volumes with 3D U-Net and Cox Proportional Hazard Neural Networks](https://mlsanity.com/paper/2202.07823/) (2022)
- [Multimodal SuperCon: Classifier for Drivers of Deforestation in Indonesia](https://mlsanity.com/paper/2207.14656/) (2022)
- [We Care Each Pixel: Calibrating on Medical Segmentation Model](https://mlsanity.com/paper/2503.05107/) (2025)
- [Exact Reformulation and Optimization for Direct Metric Optimization in Binary Imbalanced Classification](https://mlsanity.com/paper/2507.15240/) (2025)

## Abstract (arXiv)

Automatic segmentation methods are an important advancement in medical image analysis. Machine learning techniques, and deep neural networks in particular, are the state-of-the-art for most medical image segmentation tasks. Issues with class imbalance pose a significant challenge in medical datasets, with lesions often occupying a considerably smaller volume relative to the background. Loss functions used in the training of deep learning algorithms differ in their robustness to class imbalance, with direct consequences for model convergence. The most commonly used loss functions for segmentation are based on either the cross entropy loss, Dice loss or a combination of the two. We propose the Unified Focal loss, a new hierarchical framework that generalises Dice and cross entropy-based losses for handling class imbalance. We evaluate our proposed loss function on five publicly available, class imbalanced medical imaging datasets: CVC-ClinicDB, Digital Retinal Images for Vessel Extraction (DRIVE), Breast Ultrasound 2017 (BUS2017), Brain Tumour Segmentation 2020 (BraTS20) and Kidney Tumour Segmentation 2019 (KiTS19). We compare our loss function performance against six Dice or cross entropy-based loss functions, across 2D binary, 3D binary and 3D multiclass segmentation tasks, demonstrating that our proposed loss function is robust to class imbalance and consistently outperforms the other loss functions. Source code is available at: https://github.com/mlyg/unified-focal-loss