# sigmoidF1: A Smooth F1 Score Surrogate Loss for Multilabel Classification

arXiv: https://arxiv.org/abs/2108.10566 · HTML: https://mlsanity.com/paper/2108.10566/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 125 references that are in the map.
- First posted on arXiv 24 Aug 2021.

## Builds on (most cited first, 30 of 39)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [Don't Decay the Learning Rate, Increase the Batch Size](https://mlsanity.com/paper/1711.00489/) (2017)
- [CNN-RNN: A Unified Framework for Multi-label Image Classification](https://mlsanity.com/paper/1604.04573/) (2016)
- [Query2Label: A Simple Transformer Way to Multi-Label Classification](https://mlsanity.com/paper/2107.10834/) (2021)
- [TResNet: High Performance GPU-Dedicated Architecture](https://mlsanity.com/paper/2003.13630/) (2020)
- [Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification](https://mlsanity.com/paper/1702.05891/) (2017)
- [In-Place Activated BatchNorm for Memory-Optimized Training of DNNs](https://mlsanity.com/paper/1712.02616/) (2017)
- [DiSMEC - Distributed Sparse Machines for Extreme Multi-label Classification](https://mlsanity.com/paper/1609.02521/) (2016)
- [CNN: Single-label to Multi-label](https://mlsanity.com/paper/1406.5726/) (2014)
- [Large-scale Multi-label Learning with Missing Labels](https://mlsanity.com/paper/1307.5101/) (2013)
- [A no-regret generalization of hierarchical softmax to extreme multi-label classification](https://mlsanity.com/paper/1810.11671/) (2018)
- [Scalable Learning of Non-Decomposable Objectives](https://mlsanity.com/paper/1608.04802/) (2016)
- [Improving Pairwise Ranking for Multi-label Image Classification](https://mlsanity.com/paper/1704.03135/) (2017)
- [Loss Functions for Top-k Error: Analysis and Insights](https://mlsanity.com/paper/1512.00486/) (2015)
- [A Unified View of Multi-Label Performance Measures](https://mlsanity.com/paper/1609.00288/) (2016)
- [Stochastic Negative Mining for Learning with Large Output Spaces](https://mlsanity.com/paper/1810.07076/) (2018)
- [Top-k Multiclass SVM](https://mlsanity.com/paper/1511.06683/) (2015)
- [Tencent ML-Images: A Large-Scale Multi-Label Image Database for Visual Representation Learning](https://mlsanity.com/paper/1901.01703/) (2019)

## Built on by (most cited first, 23 of 23)

- [Pulse-PPG: An Open-Source Field-Trained PPG Foundation Model for Wearable Applications Across Lab and Field Settings](https://mlsanity.com/paper/2502.01108/) (2025)
- [Learning with Noisy Labels by Adaptive Gradient-Based Outlier Removal](https://mlsanity.com/paper/2306.04502/) (2023)
- [Do Machine Learning Models Learn Statistical Rules Inferred from Data?](https://mlsanity.com/paper/2303.01433/) (2023)
- [Predicting machine failures from multivariate time series: an industrial case study](https://mlsanity.com/paper/2402.17804/) (2024)
- [AnyLoss: Transforming Classification Metrics into Loss Functions](https://mlsanity.com/paper/2405.14745/) (2024)
- [EXACT: How to Train Your Accuracy](https://mlsanity.com/paper/2205.09615/) (2022)
- [Predicting O-GlcNAcylation Sites in Mammalian Proteins with Transformers and RNNs Trained with a New Loss Function](https://mlsanity.com/paper/2402.17131/) (2024)
- [Distributional Surgery for Language Model Activations](https://mlsanity.com/paper/2501.15758/) (2025)
- [Exact Reformulation and Optimization for Direct Metric Optimization in Binary Imbalanced Classification](https://mlsanity.com/paper/2507.15240/) (2025)
- [Multi-Modal Landslide Detection from Sentinel-1 SAR and Sentinel-2 Optical Imagery Using Multi-Encoder Vision Transformers and Ensemble Learning](https://mlsanity.com/paper/2604.05959/) (2026)
- [Deep Learning Approach for Aggressive Driving Behaviour Detection](https://mlsanity.com/paper/2111.04794/) (2021)
- [Simple Weak Coresets for Non-Decomposable Classification Measures](https://mlsanity.com/paper/2312.09885/) (2023)
- [EVOKE: Emotion Enabled Virtual Avatar Mapping Using Optimized Knowledge Distillation](https://mlsanity.com/paper/2401.06957/) (2024)
- [Transformer models as an efficient replacement for statistical test suites to evaluate the quality of random numbers](https://mlsanity.com/paper/2405.03904/) (2024)
- [Uncertainty-Aware Delivery Delay Duration Prediction via Multi-Task Deep Learning](https://mlsanity.com/paper/2602.20271/) (2026)
- [Reformulating van Rijsbergen's Fβ metric for weighted binary cross-entropy](https://mlsanity.com/paper/2210.16458/) (2022)
- [Aligning Multiclass Neural Network Classifier Criterion with Task Performance Metrics](https://mlsanity.com/paper/2405.20954/) (2024)
- [Tile Compression and Embeddings for Multi-Label Classification in GeoLifeCLEF 2024](https://mlsanity.com/paper/2407.06326/) (2024)
- [Semantic F1 Scores: Fair Evaluation Under Fuzzy Class Boundaries](https://mlsanity.com/paper/2509.21633/) (2025)
- [Principled Algorithms for Optimizing Generalized Metrics in Multi-Label Learning](https://mlsanity.com/paper/2605.28767/) (2026)
- [HEDGE: A Calibrated Ensemble for A/H Recognition](https://mlsanity.com/paper/2607.12176/) (2026)
- [AffectFuse: Cross-Task Feature Fusion with Temporal Modeling for Multi-Task Affective Behavior Analysis](https://mlsanity.com/paper/2607.16546/) (2026)
- [Pixels to Keys: Exploring Spatial and Motion Cues in Gameplay Inverse Dynamics](https://mlsanity.com/paper/2609.37907/) (2026)

## Abstract (arXiv)

Multiclass multilabel classification is the task of attributing multiple labels to examples via predictions. Current models formulate a reduction of the multilabel setting into either multiple binary classifications or multiclass classification, allowing for the use of existing loss functions (sigmoid, cross-entropy, logistic, etc.). Multilabel classification reductions do not accommodate for the prediction of varying numbers of labels per example and the underlying losses are distant estimates of the performance metrics. We propose a loss function, sigmoidF1, which is an approximation of the F1 score that (1) is smooth and tractable for stochastic gradient descent, (2) naturally approximates a multilabel metric, and (3) estimates label propensities and label counts. We show that any confusion matrix metric can be formulated with a smooth surrogate. We evaluate the proposed loss function on text and image datasets, and with a variety of metrics, to account for the complexity of multilabel classification evaluation. sigmoidF1 outperforms other loss functions on one text and two image datasets and several metrics. These results show the effectiveness of using inference-time metrics as loss functions for non-trivial classification problems like multilabel classification.