# Asymmetric Loss Functions and Deep Densely Connected Networks for Highly Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection

arXiv: https://arxiv.org/abs/1803.11078 · HTML: https://mlsanity.com/paper/1803.11078/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Mar 2018.

## Built on by (most cited first, 22 of 22)

- [Medical Image Segmentation Review: The success of U-Net](https://mlsanity.com/paper/2211.14830/) (2022)
- [Unified Focal loss: Generalising Dice and cross entropy-based losses to handle class imbalanced medical image segmentation](https://mlsanity.com/paper/2102.04525/) (2021)
- [A Closer Look at AUROC and AUPRC under Class Imbalance](https://mlsanity.com/paper/2401.06091/) (2024)
- [Analyzing Overfitting under Class Imbalance in Neural Networks for Image Segmentation](https://mlsanity.com/paper/2102.10365/) (2021)
- [Are we using appropriate segmentation metrics? Identifying correlates of human expert perception for CNN training beyond rolling the DICE coefficient](https://mlsanity.com/paper/2103.06205/) (2021)
- [Medical Image Segmentation with 3D Convolutional Neural Networks: A Survey](https://mlsanity.com/paper/2108.08467/) (2021)
- [Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art](https://mlsanity.com/paper/2308.16316/) (2023)
- [Multiple Sclerosis Lesion Analysis in Brain Magnetic Resonance Images: Techniques and Clinical Applications](https://mlsanity.com/paper/2104.10029/) (2021)
- [Applications of Deep Learning Techniques for Automated Multiple Sclerosis Detection Using Magnetic Resonance Imaging: A Review](https://mlsanity.com/paper/2105.04881/) (2021)
- [Longitudinal Quantitative Assessment of COVID-19 Infection Progression from Chest CTs](https://mlsanity.com/paper/2103.07240/) (2021)
- [Point detection through multi-instance deep heatmap regression for sutures in endoscopy](https://mlsanity.com/paper/2111.08468/) (2021)
- [SemSegLoss: A python package of loss functions for semantic segmentation](https://mlsanity.com/paper/2106.05844/) (2021)
- [Multiple Sclerosis Lesions Segmentation using Attention-Based CNNs in FLAIR Images](https://mlsanity.com/paper/2201.01832/) (2022)
- [PixelGame: Infrared small target segmentation as a Nash equilibrium](https://mlsanity.com/paper/2205.13124/) (2022)
- [Segment Any Medical Model Extended](https://mlsanity.com/paper/2403.18114/) (2024)
- [Are Foundation Models Ready for Industrial Defect Recognition? A Reality Check on Real-World Data](https://mlsanity.com/paper/2509.20479/) (2025)
- [Ensemble CNN and Uncertainty Modeling to Improve Automatic Identification/Segmentation of Multiple Sclerosis Lesions in Magnetic Resonance Imaging](https://mlsanity.com/paper/2108.11791/) (2021)
- [CORPS: Cost-free Rigorous Pseudo-labeling based on Similarity-ranking for Brain MRI Segmentation](https://mlsanity.com/paper/2205.09601/) (2022)
- [A Bibliography of Multiple Sclerosis Lesions Detection Methods using Brain MRIs](https://mlsanity.com/paper/2302.09516/) (2023)
- [FP-PET: Large Model, Multiple Loss And Focused Practice](https://mlsanity.com/paper/2309.12650/) (2023)
- [Component-Adaptive and Lesion-Level Supervision for Improved Small Structure Segmentation in Brain MRI](https://mlsanity.com/paper/2604.08015/) (2026)
- [Boxes2Pixels: Learning Defect Segmentation from Noisy SAM Masks](https://mlsanity.com/paper/2604.11162/) (2026)

## Abstract (arXiv)

Fully convolutional deep neural networks have been asserted to be fast and precise frameworks with great potential in image segmentation. One of the major challenges in training such networks raises when data is unbalanced, which is common in many medical imaging applications such as lesion segmentation where lesion class voxels are often much lower in numbers than non-lesion voxels. A trained network with unbalanced data may make predictions with high precision and low recall, being severely biased towards the non-lesion class which is particularly undesired in most medical applications where FNs are more important than FPs. Various methods have been proposed to address this problem, more recently similarity loss functions and focal loss. In this work we trained fully convolutional deep neural networks using an asymmetric similarity loss function to mitigate the issue of data imbalance and achieve much better tradeoff between precision and recall. To this end, we developed a 3D FC-DenseNet with large overlapping image patches as input and an asymmetric similarity loss layer based on Tversky index (using Fbeta scores). We used large overlapping image patches as inputs for intrinsic and extrinsic data augmentation, a patch selection algorithm, and a patch prediction fusion strategy using B-spline weighted soft voting to account for the uncertainty of prediction in patch borders. We applied this method to MS lesion segmentation based on two different datasets of MSSEG and ISBI longitudinal MS lesion segmentation challenge, where we achieved top performance in both challenges. Our network trained with focal loss ranked first according to the ISBI challenge overall score and resulted in the lowest reported lesion false positive rate among all submitted methods. Our network trained with the asymmetric similarity loss led to the lowest surface distance and the best lesion true positive rate.