mlsanityData to 7 Oct 2026

Reformulating van Rijsbergen's Fβ metric for weighted binary cross-entropy

arXiv 2210.16458 ↗First posted 29 Oct 2022stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The separation of performance metrics from gradient based loss functions may not always give optimal results and may miss vital aggregate information. This paper investigates incorporating a performance metric alongside differentiable loss functions to inform training outcomes. The goal is to guide model performance and interpretation by assuming statistical distributions on this performance metric for dynamic weighting. The focus is on van Rijsbergens Fβ metric -- a popular choice for gauging classification performance. Through distributional assumptions on the Fβ, an intermediary link can be established to the standard binary cross-entropy via dynamic penalty weights. First, the Fβ metric is reformulated to facilitate assuming statistical distributions with accompanying proofs for the cumulative density function. These probabilities are used within a knee curve algorithm to find an optimal β or βopt. This βopt is used as a weight or penalty in the proposed weighted binary cross-entropy. Experimentation on publicly available data along with benchmark analysis mostly yields better and interpretable results as compared to the baseline for both imbalanced and balanced classes. For example, for the IMDB text data with known labeling errors, a 14% boost in F1 score is shown. The results also reveal commonalities between the penalty model families derived in this paper and the suitability of recall-centric or precision-centric parameters used in the optimization. The flexibility of this methodology can enhance interpretation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 62 references that are in the map. First posted on arXiv 29 Oct 2022.

Builds on

17 of 62 references.

References · 17

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  4. 2014Deep Learning in Neural Networks: An Overviewcited by 765
  5. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 580
  6. 2021Deep Neural Networks and Tabular Data: A Surveycited by 489
  7. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 388
  8. 2019Dice Loss for Data-imbalanced NLP Taskscited by 180
  9. 2020The Real-World-Weight Cross-Entropy Loss Function: Modeling the Costs of Mislabelingcited by 64
  10. 2022Annotation Error Detection: Analyzing the Past and Present for a More Coherent Futurecited by 45
  11. 2018Localization Recall Precision (LRP): A New Performance Metric for Object Detectioncited by 31
  12. 2020A Ranking-based, Balanced Loss Function Unifying Classification and Localisation in Object Detectioncited by 25
5 more
  1. 2021sigmoidF1: A Smooth F1 Score Surrogate Loss for Multilabel Classificationcited by 23
  2. 2021Striking the Right Balance: Recall Loss for Semantic Segmentationcited by 19
  3. 2019Testing DNN Image Classifiers for Confusion & Bias Errorscited by 17
  4. 2021A surrogate loss function for optimization of Fβ score in binary classification with imbalanced datacited by 5
  5. 2021Maximum F1-score training for end-to-end mispronunciation detection and diagnosis of L2 English speechcited by 4

Built on by

Not cited yet.