mlsanityData to 7 Oct 2026

Learning to Reject with a Fixed Predictor: Application to Decontextualization

arXiv 2301.09044 ↗First posted 22 Jan 2023cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

We study the problem of classification with a reject option for a fixed predictor, applicable in natural language processing. We introduce a new problem formulation for this scenario, and an algorithm minimizing a new surrogate loss function. We provide a complete theoretical analysis of the surrogate loss function with a strong H-consistency guarantee. For evaluation, we choose the decontextualization task, and provide a manually-labelled dataset of 2mathord,000 examples. Our algorithm significantly outperforms the baselines considered, with a ~!!25% improvement in coverage when halving the error rate, which is only ~!! 3 % away from the theoretical limit.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 29 of its 61 references that are in the map. First posted on arXiv 22 Jan 2023.

Builds on

29 of 61 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  7. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  8. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  9. 2019How Can We Know What Language Models Know?cited by 953
  10. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  11. 2017Selective Classification for Deep Neural Networkscited by 799
  12. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
17 more
  1. 2018Deep Learning for Classical Japanese Literaturecited by 415
  2. 2020Selective Question Answering under Domain Shiftcited by 228
  3. 2022Scaling Up Models and Data with textttt5x and textttseqiocited by 187
  4. 2018Understanding Measures of Uncertainty for Adversarial Example Detectioncited by 177
  5. 2021Machine Learning with a Reject Option: A surveycited by 107
  6. 2021Decontextualization: Making Sentences Stand-Alonecited by 101
  7. 2020Controlled Hallucinations: Learning to Generate Faithfully from Noisy Datacited by 83
  8. 2019Calibration of Encoder Decoder Models for Neural Machine Translationcited by 67
  9. 2020Classification with Rejection Based on Cost-sensitive Classificationcited by 55
  10. 2021Calibration and Consistency of Adversarial Surrogate Lossescited by 47
  11. 2021Knowing More About Questions Can Help: Improving Calibration in Question Answeringcited by 47
  12. 2019On the Calibration of Multiclass Classification with Rejectioncited by 41
  13. 2018Confidence Modeling for Neural Semantic Parsingcited by 33
  14. 2021A Finer Calibration Analysis for Adversarial Robustnesscited by 26
  15. 2020Towards Improving Selective Prediction Ability of NLP Systemscited by 19
  16. 2021Will this Question be Answered? Question Filtering via Answer Model Distillation for Efficient Question Answeringcited by 19
  17. 2017The Relationship Between Agnostic Selective Classification Active Learning and the Disagreement Coefficientcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 12; 2025: 8; 2026: 8.

32 papers.

Sort:

  1. 2024Language Models with Conformal Factuality Guaranteescited by 128
  2. 2024Learning to Route LLMs with Confidence Tokenscited by 66
  3. 2024Learning to Decode Collaboratively with Multiple Language Modelscited by 61
  4. 2023Predictor-Rejector Multi-Class Abstention: Theoretical Analysis and Algorithmscited by 36
  5. 2023Theoretically Grounded Loss Functions and Algorithms for Score-Based Multi-Class Abstentioncited by 32
  6. 2024Regression with Multi-Expert Deferralcited by 29
  7. 2024Realizable H-Consistent and Bayes-Consistent Loss Functions for Learning to Defercited by 26
  8. 2024A Universal Growth Rate for Learning with Smooth Surrogate Lossescited by 24
  9. 2024Cardinality-Aware Set Prediction and Top-k Classificationcited by 23
  10. 2024Multi-Label Learning with Stronger Consistency Guaranteescited by 22
  11. 2024H-Consistency Guarantees for Regressioncited by 21
  12. 2025Conformal Linguistic Calibration: Trading-off between Factuality and Specificitycited by 20
20 more
  1. 2024Enhanced H-Consistency Boundscited by 18
  2. 2025Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees for Learning to Defercited by 16
  3. 2023Selective Generation for Controllable Language Modelscited by 15
  4. 2024Optimal Query Allocation in Extractive QA with LLMs: A Learning-to-Defer Framework with Theoretical Guaranteescited by 15
  5. 2025Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Datacited by 15
  6. 2023Learning to Make Adherence-Aware Advicecited by 13
  7. 2025Budgeted Multiple-Expert Deferralcited by 11
  8. 2025Improved Balanced Classification with Theoretically Grounded Loss Functionscited by 9
  9. 2025Why Ask One When You Can Ask k? Learning-to-Defer to the Top-k Expertscited by 6
  10. 2025Principled Algorithms for Optimizing Generalized Metrics in Binary Classificationcited by 6
  11. 2024Top-k Classification and Cardinality-Aware Predictioncited by 5
  12. 2025Adversarial Robustness in One-Stage Learning-to-Defercited by 5
  13. 2026A Theoretical Framework for Modular Learning of Robust Generative Modelscited by 4
  14. 2026Optimized Deferral for Imbalanced Settingscited by 2
  15. 2026Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Predictioncited by 2
  16. 2026Selective Prior Synchronization via SYNC Losscited by 0
  17. 2026Flexible Routing via Uncertainty Decompositioncited by 0
  18. 2026Online Learning-to-Defer with Varying Expertscited by 0
  19. 2026Principled Algorithms for Optimizing Generalized Metrics in Multi-Label Learningcited by 0
  20. 2026Human-AI Teaming Through the Lens of Calibrationcited by 0