mlsanityData to 7 Oct 2026

Theoretically Grounded Loss Functions and Algorithms for Score-Based Multi-Class Abstention

arXiv 2310.14770 ↗First posted 23 Oct 2023cs.LG stat.MLcited by 32

Not on the map. Why?

Abstract (arXiv)

Learning with abstention is a key scenario where the learner can abstain from making a prediction at some cost. In this paper, we analyze the score-based formulation of learning with abstention in the multi-class classification setting. We introduce new families of surrogate losses for the abstention loss function, which include the state-of-the-art surrogate losses in the single-stage setting and a novel family of loss functions in the two-stage setting. We prove strong non-asymptotic and hypothesis set-specific consistency guarantees for these surrogate losses, which upper-bound the estimation error of the abstention loss function in terms of the estimation error of the surrogate loss. Our bounds can help compare different score-based surrogates and guide the design of novel abstention algorithms by minimizing the proposed surrogate losses. We experimentally evaluate our new algorithms on CIFAR-10, CIFAR-100, and SVHN datasets and the practical significance of our new surrogate losses and two-stage abstention algorithms. Our results also show that the relative performance of the state-of-the-art score-based surrogate losses can vary across datasets.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 34 of its 112 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

34 of 112 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  5. 2016Wide Residual Networkscited by 3,057
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  7. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
  8. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  9. 2018Robustness May Be at Odds with Accuracycited by 873
  10. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
  11. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 333
  12. 2020Consistent Estimators for Learning to Defer to an Expertcited by 262
22 more
  1. 2017Predict Responsibly: Improving Fairness and Accuracy by Learning to Defercited by 228
  2. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 206
  3. 2020Learning to Complement Humanscited by 123
  4. 2019The Algorithmic Automation Problem: Prediction, Triage, and Human Effortcited by 101
  5. 2020Controlled Hallucinations: Learning to Generate Faithfully from Noisy Datacited by 83
  6. 2021Differentiable Learning Under Triagecited by 74
  7. 2019Deep Gamblers: Learning to Abstain with Portfolio Theorycited by 69
  8. 2022Calibrated Learning to Defer with One-vs-All Classifierscited by 68
  9. 2023Who Should Predict? Exact Algorithms For Learning to Defer to Humanscited by 68
  10. 2018Direct Uncertainty Prediction for Medical Second Opinionscited by 61
  11. 2022Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensemblescited by 59
  12. 2021Calibration and Consistency of Adversarial Surrogate Lossescited by 47
  13. 2020Selective Classification via One-Sided Predictioncited by 37
  14. 2023Learning to Reject with a Fixed Predictor: Application to Decontextualizationcited by 32
  15. 2023Revisiting Discriminative vs. Generative Classifiers: Theory and Implicationscited by 29
  16. 2021Classification with abstention but without disparitiescited by 27
  17. 2023Ranking with Abstentioncited by 27
  18. 2021A Finer Calibration Analysis for Adversarial Robustnesscited by 26
  19. 2021Exponential Savings in Agnostic Active Learning through Abstentioncited by 23
  20. 2023Learning to Make Adherence-Aware Advicecited by 13
  21. 2014A Compression Technique for Analyzing Disagreement-Based Active Learningcited by 11
  22. 2016The Extended Littlestone's Dimension for Learning with Mistakes and Abstentionscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 12; 2026: 8.

32 papers.

Sort:

  1. 2024Regression with Multi-Expert Deferralcited by 29
  2. 2024Realizable H-Consistent and Bayes-Consistent Loss Functions for Learning to Defercited by 26
  3. 2024A Universal Growth Rate for Learning with Smooth Surrogate Lossescited by 24
  4. 2024Cardinality-Aware Set Prediction and Top-k Classificationcited by 23
  5. 2024Multi-Label Learning with Stronger Consistency Guaranteescited by 22
  6. 2024H-Consistency Guarantees for Regressioncited by 21
  7. 2024Enhanced H-Consistency Boundscited by 18
  8. 2024A Two-Stage Learning-to-Defer Approach for Multi-Task Learningcited by 16
  9. 2025Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees for Learning to Defercited by 16
  10. 2024Cascade-Aware Training of Language Modelscited by 15
  11. 2024Optimal Query Allocation in Extractive QA with LLMs: A Learning-to-Defer Framework with Theoretical Guaranteescited by 15
  12. 2025Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Datacited by 15
20 more
  1. 2025Budgeted Multiple-Expert Deferralcited by 11
  2. 2025Improved Balanced Classification with Theoretically Grounded Loss Functionscited by 9
  3. 2024Hierarchical Selective Classificationcited by 7
  4. 2025Beyond Tsybakov: Model Margin Noise and H-Consistency Boundscited by 7
  5. 2025Why Ask One When You Can Ask k? Learning-to-Defer to the Top-k Expertscited by 6
  6. 2025Principled Algorithms for Optimizing Generalized Metrics in Binary Classificationcited by 6
  7. 2024Top-k Classification and Cardinality-Aware Predictioncited by 5
  8. 2025Adversarial Robustness in One-Stage Learning-to-Defercited by 5
  9. 2025Learning to Partially Defer for Sequencescited by 4
  10. 2026A Theoretical Framework for Modular Learning of Robust Generative Modelscited by 4
  11. 2025Learning to Help in Multi-Class Settingscited by 2
  12. 2026Optimized Deferral for Imbalanced Settingscited by 2
  13. 2026Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Predictioncited by 2
  14. 2025From Deferral to Learning: Online In-Context Knowledge Distillation for LLM Cascadescited by 1
  15. 2026Consistent Learning-to-Defer with Expert-Conditional Advicecited by 1
  16. 2026Calibrating conditional riskcited by 1
  17. 2025On the Bayes Inconsistency of Disagreement Discrepancy Surrogatescited by 0
  18. 2026Flexible Routing via Uncertainty Decompositioncited by 0
  19. 2026Online Learning-to-Defer with Varying Expertscited by 0
  20. 2026Principled Algorithms for Optimizing Generalized Metrics in Multi-Label Learningcited by 0