Online Learning-to-Defer with Varying Experts
Not on the map. Why?
Abstract (arXiv)
Learning-to-Defer (L2D) methods route each query either to a predictive model or to external experts. Real-world deployments require handling streaming data, changing expert availability, shifting expert reliability, and feedback observed only for the selected action. We introduce an online multiclass L2D algorithm that combines queried-action bandit feedback with a dynamically varying pool of experts. Let N=n+ne, let B bound the Frobenius norm of the linear score matrix, and let ρ bound the augmented input norm. Assuming linear calibration and zero surrogate minimizability gap for the projected comparator class, our method achieves expected true-deferral regret O((BN3/2ρ+1)T2/3), improving to O(BN3/2ρ√ T+B2N3ρ2) under a concentrated-score condition. The analysis combines an online mathcal H-consistency transfer bound with projected online convex optimization. Experiments on synthetic and real-world datasets demonstrate selective routing under varying expert availability and reliability.
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Builds on 61 of its 117 references that are in the map. First posted on arXiv 12 May 2026.
Builds on
61 of 117 references.
References · 61
- 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,862
- 2017Selective Classification for Deep Neural Networkscited by 825
- 2019Online Learning: A Modern Introduction Using Convex Optimizationcited by 412
- 2020Consistent Estimators for Learning to Defer to an Expertcited by 268
- 2017Predict Responsibly: Improving Fairness and Accuracy by Learning to Defercited by 232
- 2019Human uncertainty makes classification more robustcited by 231
- 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 207
- 2014Taming the Monster: A Fast and Simple Algorithm for Contextual Banditscited by 200
- 2021Towards Unbiased and Accurate Deferral to Multiple Expertscited by 71
- 2023Who Should Predict? Exact Algorithms For Learning to Defer to Humanscited by 70
- 2022Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensemblescited by 59
- 2021Calibration and Consistency of Adversarial Surrogate Lossescited by 47
38 more
- 2022Sample Efficient Learning of Predictors that Complement Humanscited by 46
- 2018Logistic Regression: The Importance of Being Impropercited by 40
- 2021Combining Human Predictions with Model Probabilities via Confusion Matrices and Calibrationcited by 40
- 2022Forming Effective Human-AI Teams: Building Machine Learning Models that Complement the Capabilities of Multiple Expertscited by 40
- 2023Principled Approaches for Learning to Defer with Multiple Expertscited by 38
- 2023Predictor-Rejector Multi-Class Abstention: Theoretical Analysis and Algorithmscited by 36
- 2023Learning to Reject with a Fixed Predictor: Application to Decontextualizationcited by 32
- 2023Theoretically Grounded Loss Functions and Algorithms for Score-Based Multi-Class Abstentioncited by 32
- 2024Regression with Multi-Expert Deferralcited by 29
- 2017Online Learning with Abstentioncited by 28
- 2023Ranking with Abstentioncited by 27
- 2023In Defense of Softmax Parametrization for Calibrated and Consistent Learning to Defercited by 26
- 2024Learning to Defer to a Population: A Meta-Learning Approachcited by 26
- 2024Realizable H-Consistent and Bayes-Consistent Loss Functions for Learning to Defercited by 26
- 2023Learning to Defer with Limited Expert Predictionscited by 25
- 2013The price of bandit information in multiclass online classificationcited by 24
- 2024A Universal Growth Rate for Learning with Smooth Surrogate Lossescited by 24
- 2024Cardinality-Aware Set Prediction and Top-k Classificationcited by 23
- 2024Multi-Label Learning with Stronger Consistency Guaranteescited by 22
- 2024H-Consistency Guarantees for Regressioncited by 21
- 2020Fast Rates for Online Prediction with Abstentioncited by 19
- 2024Enhanced H-Consistency Boundscited by 18
- 2020Efficient improper learning for online logistic regressioncited by 16
- 2024A Two-Stage Learning-to-Defer Approach for Multi-Task Learningcited by 16
- 2025Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guaranteescited by 16
- 2025Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees for Learning to Defercited by 16
- 2024Optimal Query Allocation in Extractive QA with LLMs: A Learning-to-Defer Framework with Theoretical Guaranteescited by 15
- 2025Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Datacited by 15
- 2021Beyond Bandit Feedback in Online Multiclass Classificationcited by 14
- 2022Counterfactual Inference of Second Opinionscited by 14
- 2024A Causal Framework for Evaluating Deferring Systemscited by 13
- 2021Mixability made efficient: Fast online multiclass logistic regressioncited by 12
- 2025Budgeted Multiple-Expert Deferralcited by 12
- 2021Efficient Methods for Online Multiclass Logistic Regressioncited by 10
- 2020Exploiting the Surrogate Gap in Online Multiclass Classificationcited by 9
- 2025Improved Balanced Classification with Theoretically Grounded Loss Functionscited by 9
- 2025Theory and Algorithms for Learning with Multi-Class Abstention and Multi-Expert Deferralcited by 8
- 2025Beyond Tsybakov: Model Margin Noise and H-Consistency Boundscited by 7
50 most cited of 61 All 61 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.