# Jaccard Metric Losses: Optimizing the Jaccard Index with Soft Labels

arXiv: https://arxiv.org/abs/2302.05666 · HTML: https://mlsanity.com/paper/2302.05666/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 68 of its 93 references that are in the map.
- First posted on arXiv 11 Feb 2023.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 30 of 68)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions](https://mlsanity.com/paper/2211.05778/) (2022)
- [Measuring Calibration in Deep Learning](https://mlsanity.com/paper/1904.01685/) (2019)
- [Decoupled Knowledge Distillation](https://mlsanity.com/paper/2203.08679/) (2022)
- [MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers](https://mlsanity.com/paper/2012.00759/) (2020)
- [Slimmable Neural Networks](https://mlsanity.com/paper/1812.08928/) (2018)

## Built on by (most cited first, 12 of 12)

- [Revisiting Evaluation Metrics for Semantic Segmentation: Optimization and Evaluation of Fine-grained Intersection over Union](https://mlsanity.com/paper/2310.19252/) (2023)
- [Dice Semimetric Losses: Optimizing the Dice Score with Soft Labels](https://mlsanity.com/paper/2303.16296/) (2023)
- [SiNGR: Brain Tumor Segmentation via Signed Normalized Geodesic Transform Regression](https://mlsanity.com/paper/2405.16813/) (2024)
- [Small molecule retrieval from tandem mass spectrometry: what are we optimizing for?](https://mlsanity.com/paper/2602.16507/) (2026)
- [ChatGPT-Crawler: Find out if ChatGPT really knows what it's talking about](https://mlsanity.com/paper/2304.03325/) (2023)
- [Federated Learning With L0 Constraint Via Probabilistic Gates For Sparsity](https://mlsanity.com/paper/2512.23071/) (2025)
- [Image-level Regression for Uncertainty-aware Retinal Image Segmentation](https://mlsanity.com/paper/2405.16815/) (2024)
- [TipSegNet: Fingertip Segmentation in Contactless Fingerprint Imaging](https://mlsanity.com/paper/2501.05076/) (2025)
- [From Text to Talent: A Pipeline for Extracting Insights from Candidate Profiles](https://mlsanity.com/paper/2503.17438/) (2025)
- [RangeSAM: On the Potential of Visual Foundation Models for Range-View represented LiDAR segmentation](https://mlsanity.com/paper/2509.15886/) (2025)
- [Complementary Text-Guided Attention for Zero-Shot Adversarial Robustness](https://mlsanity.com/paper/2603.18598/) (2026)
- [Mosaic: Compositional Multi-Concept Erasure via Vector Field Blending](https://mlsanity.com/paper/2605.25574/) (2026)

## Abstract (arXiv)

Intersection over Union (IoU) losses are surrogates that directly optimize the Jaccard index. Leveraging IoU losses as part of the loss function have demonstrated superior performance in semantic segmentation tasks compared to optimizing pixel-wise losses such as the cross-entropy loss alone. However, we identify a lack of flexibility in these losses to support vital training techniques like label smoothing, knowledge distillation, and semi-supervised learning, mainly due to their inability to process soft labels. To address this, we introduce Jaccard Metric Losses (JMLs), which are identical to the soft Jaccard loss in standard settings with hard labels but are fully compatible with soft labels. We apply JMLs to three prominent use cases of soft labels: label smoothing, knowledge distillation and semi-supervised learning, and demonstrate their potential to enhance model accuracy and calibration. Our experiments show consistent improvements over the cross-entropy loss across 4 semantic segmentation datasets (Cityscapes, PASCAL VOC, ADE20K, DeepGlobe Land) and 13 architectures, including classic CNNs and recent vision transformers. Remarkably, our straightforward approach significantly outperforms state-of-the-art knowledge distillation and semi-supervised learning methods. The code is available at hrefhttps://github.com/zifuwanggg/JDTLosseshttps://github.com/zifuwanggg/JDTLosses.