mlsanityData to 7 Oct 2026

Improved Balanced Classification with Theoretically Grounded Loss Functions

arXiv 2512.23947 ↗First posted 30 Dec 2025cs.LG stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

The balanced loss is a widely adopted objective for multi-class classification under class imbalance. By assigning equal importance to all classes, regardless of their frequency, it promotes fairness and ensures that minority classes are not overlooked. However, directly minimizing the balanced classification loss is typically intractable, which makes the design of effective surrogate losses a central question. This paper introduces and studies two advanced surrogate loss families: Generalized Logit-Adjusted (GLA) loss functions and Generalized Class-Aware weighted (GCA) losses. GLA losses generalize Logit-Adjusted losses, which shift logits based on class priors, to the broader general cross-entropy loss family. GCA loss functions extend the standard class-weighted losses, which scale losses inversely by class frequency, by incorporating class-dependent confidence margins and extending them to the general cross-entropy family. We present a comprehensive theoretical analysis of consistency for both loss families. We show that GLA losses are Bayes-consistent, but only H-consistent for complete (i.e., unbounded) hypothesis sets. Moreover, their H-consistency bounds depend inversely on the minimum class probability, scaling at least as 1/mathsf pmin. In contrast, GCA losses are H-consistent for any hypothesis set that is bounded or complete, with H-consistency bounds that scale more favorably as 1/√mathsf pmin, offering significantly stronger theoretical guarantees in imbalanced settings. We report the results of experiments demonstrating that, empirically, both the GCA losses with calibrated class-dependent confidence margins and GLA losses can greatly outperform straightforward class-weighted losses as well as the LA losses. GLA generally performs slightly better in common benchmarks, whereas GCA exhibits a slight edge in highly imbalanced settings.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 90 of its 153 references that are in the map. First posted on arXiv 30 Dec 2025.

Builds on

90 of 153 references.

References · 90

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  5. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  6. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
  7. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  8. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  9. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  10. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  11. 2020Long-tail learning via logit adjustmentcited by 605
  12. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 510
38 more
  1. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  2. 2019Agnostic Federated Learningcited by 451
  3. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
  4. 2021Deep Long-Tailed Learning: A Surveycited by 416
  5. 2019BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognitioncited by 403
  6. 2020Balanced Meta-Softmax for Long-Tailed Visual Recognitioncited by 397
  7. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  8. 2020Equalization Loss for Long-Tailed Object Recognitioncited by 285
  9. 2020Long-tailed Recognition by Routing Diverse Distribution-Aware Expertscited by 270
  10. 2020Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effectcited by 256
  11. 2018What is the Effect of Importance Weighting in Deep Learning?cited by 250
  12. 2021Parametric Contrastive Learningcited by 217
  13. 2020Rethinking the Value of Labels for Improving Class-Imbalanced Learningcited by 216
  14. 2021Improving Calibration for Long-Tailed Recognitioncited by 212
  15. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 206
  16. 2020Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classificationcited by 175
  17. 2020Disentangling Label Distribution for Long-tailed Visual Recognitioncited by 172
  18. 2020Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspectivecited by 143
  19. 2016A vector-contraction inequality for Rademacher complexitiescited by 141
  20. 2016Range Loss for Deep Face Recognition with Long-tailcited by 111
  21. 2022Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?cited by 100
  22. 2021ResLT: Residual Learning for Long-tailed Recognitioncited by 96
  23. 2021Self-Supervised Aggregation of Diverse Experts for Test-Agnostic Long-Tailed Recognitioncited by 95
  24. 2019Striking the Right Balance with Uncertaintycited by 79
  25. 2020Identifying and Compensating for Feature Deviation in Imbalanced Deep Learningcited by 76
  26. 2021Label-Imbalanced and Group-Sensitive Classification under Overparameterizationcited by 74
  27. 2021Learning Fast Sample Re-weighting Without Reward Datacited by 69
  28. 2021RSG: A Simple but Effective Module for Learning Imbalanced Datasetscited by 61
  29. 2023Long-Tail Learning with Foundation Model: Heavy Fine-Tuning Hurtscited by 58
  30. 2016AdaNet: Adaptive Structural Learning of Artificial Neural Networkscited by 54
  31. 2020Posterior Re-calibration for Imbalanced Datasetscited by 53
  32. 2017Learning with Average Top-k Losscited by 52
  33. 2019Adjusting Decision Boundary for Class Imbalanced Learningcited by 52
  34. 2022AutoBalance: Optimized Loss Functions for Imbalanced Datacited by 49
  35. 2021Calibration and Consistency of Adversarial Surrogate Lossescited by 47
  36. 2023Principled Approaches for Learning to Defer with Multiple Expertscited by 38
  37. 2023Predictor-Rejector Multi-Class Abstention: Theoretical Analysis and Algorithmscited by 36
  38. 2023How Re-sampling Helps for Long-Tail Learning?cited by 36

50 most cited of 90 All 90 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026A Theoretical Framework for Modular Learning of Robust Generative Modelscited by 4
  2. 2026Optimized Deferral for Imbalanced Settingscited by 2
  3. 2026Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Predictioncited by 2
  4. 2026Online Learning-to-Defer with Varying Expertscited by 0
  5. 2026Principled Algorithms for Optimizing Generalized Metrics in Multi-Label Learningcited by 0
  6. 2026Angular Gaussian Supervised Contrastive Learning for Long-Tailed Electrocardiogram Arrhythmia Diagnosiscited by 0
  7. 2026Generalised Balanced Softmax: A Finite-Data Perspective on Logit Adjustment for Long-Tailed Recognitioncited by 0
  8. 2026Bounded Adjustment with Reliability-Guided Embedding for Imbalanced Learning with Noisy Labelscited by 0
  9. 2026Age of Learning: Temporal Persistence of Prediction Errors as a Learning Signalcited by 0