mlsanityData to 14 Oct 2026

SCRIB: Set-classifier with Class-specific Risk Bounds for Blackbox Models

arXiv 2103.03945 ↗First posted 5 Mar 2021cs.LG stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

Despite deep learning (DL) success in classification problems, DL classifiers do not provide a sound mechanism to decide when to refrain from predicting. Recent works tried to control the overall prediction risk with classification with rejection options. However, existing works overlook the different significance of different classes. We introduce Set-classifier with Class-specific RIsk Bounds (SCRIB) to tackle this problem, assigning multiple labels to each example. Given the output of a black-box model on the validation set, SCRIB constructs a set-classifier that controls the class-specific prediction risks with a theoretical guarantee. The key idea is to reject when the set classifier returns more than one label. We validated SCRIB on several medical applications, including sleep staging on electroencephalogram (EEG) data, X-ray COVID image classification, and atrial fibrillation detection based on electrocardiogram (ECG) data. SCRIB obtained desirable class-specific risks, which are 35%-88% closer to the target risks than baseline methods.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 22 of its 61 references that are in the map. First posted on arXiv 5 Mar 2021.

Builds on

22 of 61 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2017On Calibration of Modern Neural Networkscited by 4,960
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  7. 2017Selective Classification for Deep Neural Networkscited by 852
  8. 2016Distribution-Free Predictive Inference For Regressioncited by 668
  9. 2015Weight Uncertainty in Neural Networkscited by 554
  10. 2015Deep Kernel Learningcited by 445
  11. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 366
  12. 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 364
10 more
  1. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 350
  2. 2019Verified Uncertainty Calibrationcited by 309
  3. 2016Least Ambiguous Set-Valued Classifiers with Bounded Error Levelscited by 272
  4. 2018To Trust Or Not To Trust A Classifiercited by 207
  5. 2019Addressing Failure Prediction by Learning Model Confidencecited by 186
  6. 2020Confidence-Aware Learning for Deep Neural Networkscited by 113
  7. 2019Non-Parametric Calibration for Classificationcited by 57
  8. 2019MINA: Multilevel Knowledge-Guided Attention for Modeling Electrocardiography Signalscited by 21
  9. 2020FLANNEL: Focal Loss Based Neural Network Ensemble for COVID-19 Detectioncited by 6
  10. 2019Efficient Set-Valued Prediction in Multi-Class Classificationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 1; 2023: 2; 2024: 3; 2025: 0; 2026: 2.

10 papers.

  1. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 293
  2. 2021Machine Learning with a Reject Option: A surveycited by 109
  3. 2024Mitigating LLM Hallucinations via Conformal Abstentioncited by 59
  4. 2022Calibrated Selective Classificationcited by 33
  5. 2021Locally Valid and Discriminative Prediction Intervals for Deep Learning Modelscited by 24
  6. 2024Contextualized Sequence Likelihood: Enhanced Confidence Scores for Natural Language Generationcited by 16
  7. 2024CSS: Contrastive Semantic Similarity for Uncertainty Quantification of LLMscited by 4
  8. 2023Fast Online Value-Maximizing Prediction Sets with Conformal Cost Controlcited by 3
  9. 2026PyHealth 2.0: A Comprehensive Open-Source Toolkit for Accessible and Reproducible Clinical Deep Learningcited by 2
  10. 2026COVER: Learning to Accept More in Selective Sleep Stagingcited by 0