mlsanityData to 14 Oct 2026

Regression with Multi-Expert Deferral

arXiv 2403.19494 ↗First posted 28 Mar 2024cs.LG stat.MLcited by 29

Not on the map. Why?

Abstract (arXiv)

Learning to defer with multiple experts is a framework where the learner can choose to defer the prediction to several experts. While this problem has received significant attention in classification contexts, it presents unique challenges in regression due to the infinite and continuous nature of the label space. In this work, we introduce a novel framework of regression with deferral, which involves deferring the prediction to multiple experts. We present a comprehensive analysis for both the single-stage scenario, where there is simultaneous learning of predictor and deferral functions, and the two-stage scenario, which involves a pre-trained predictor with a learned deferral function. We introduce new surrogate loss functions for both scenarios and prove that they are supported by H-consistency bounds. These bounds provide consistency guarantees that are stronger than Bayes consistency, as they are non-asymptotic and hypothesis set-specific. Our framework is versatile, applying to multiple experts, accommodating any bounded regression losses, addressing both instance-dependent and label-dependent costs, and supporting both single-stage and two-stage methods. A by-product is that our single-stage formulation includes the recent regression with abstention framework (Cheng et al., 2023) as a special case, where only a single expert, the squared loss and a label-independent cost are considered. Minimizing our proposed loss functions directly leads to novel algorithms for regression with deferral. We report the results of extensive experiments showing the effectiveness of our proposed algorithms.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 26 of its 71 references that are in the map. First posted on arXiv 28 Mar 2024.

Builds on

26 of 71 references.

References · 26

  1. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  2. 2022Emergent Abilities of Large Language Modelscited by 2,512
  3. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 350
  4. 2020Consistent Estimators for Learning to Defer to an Expertcited by 271
  5. 2017Predict Responsibly: Improving Fairness and Accuracy by Learning to Defercited by 233
  6. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 210
  7. 2021Differentiable Learning Under Triagecited by 76
  8. 2021Towards Unbiased and Accurate Deferral to Multiple Expertscited by 72
  9. 2022Calibrated Learning to Defer with One-vs-All Classifierscited by 69
  10. 2022Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensemblescited by 60
  11. 2021Calibration and Consistency of Adversarial Surrogate Lossescited by 47
  12. 2019Regression Under Human Assistancecited by 40
14 more
  1. 2021Combining Human Predictions with Model Probabilities via Confusion Matrices and Calibrationcited by 40
  2. 2022Forming Effective Human-AI Teams: Building Machine Learning Models that Complement the Capabilities of Multiple Expertscited by 40
  3. 2023Principled Approaches for Learning to Defer with Multiple Expertscited by 38
  4. 2023Predictor-Rejector Multi-Class Abstention: Theoretical Analysis and Algorithmscited by 36
  5. 2021Selective Regression Under Fairness Criteriacited by 34
  6. 2023Learning to Reject with a Fixed Predictor: Application to Decontextualizationcited by 32
  7. 2023Theoretically Grounded Loss Functions and Algorithms for Score-Based Multi-Class Abstentioncited by 32
  8. 2020Regression with reject option and application to kNNcited by 30
  9. 2023Revisiting Discriminative vs. Generative Classifiers: Theory and Implicationscited by 29
  10. 2023Ranking with Abstentioncited by 27
  11. 2024Learning to Defer to a Population: A Meta-Learning Approachcited by 27
  12. 2021A Finer Calibration Analysis for Adversarial Robustnesscited by 26
  13. 2022Counterfactual Inference of Second Opinionscited by 14
  14. 2023Regression with Cost-based Rejectioncited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 12; 2026: 10.

29 papers.

Sort:

By area:

  1. 2025Universal Model Routing for Efficient LLM Inferencecited by 86
  2. 2024Realizable H-Consistent and Bayes-Consistent Loss Functions for Learning to Defercited by 26
  3. 2024A Universal Growth Rate for Learning with Smooth Surrogate Lossescited by 24
  4. 2024Cardinality-Aware Set Prediction and Top-k Classificationcited by 23
  5. 2024Multi-Label Learning with Stronger Consistency Guaranteescited by 22
  6. 2024Enhanced H-Consistency Boundscited by 18
  7. 2024A Two-Stage Learning-to-Defer Approach for Multi-Task Learningcited by 17
  8. 2025Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guaranteescited by 16
  9. 2025Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees for Learning to Defercited by 16
  10. 2024Optimal Query Allocation in Extractive QA with LLMs: A Learning-to-Defer Framework with Theoretical Guaranteescited by 15
  11. 2025Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Datacited by 15
  12. 2025Budgeted Multiple-Expert Deferralcited by 13
17 more
  1. 2025Improved Balanced Classification with Theoretically Grounded Loss Functionscited by 9
  2. 2025Why Ask One When You Can Ask k? Learning-to-Defer to the Top-k Expertscited by 7
  3. 2025Principled Algorithms for Optimizing Generalized Metrics in Binary Classificationcited by 6
  4. 2026Learning-to-Defer in Non-Stationary Time Series via Switching State-Space Modelscited by 6
  5. 2025Adversarial Robustness in One-Stage Learning-to-Defercited by 5
  6. 2025Learning to Partially Defer for Sequencescited by 4
  7. 2026A Theoretical Framework for Modular Learning of Robust Generative Modelscited by 4
  8. 2026Consistent Learning-to-Defer with Expert-Conditional Advicecited by 3
  9. 2026Fatigue-Aware Learning to Defer via Constrained Optimisationcited by 3
  10. 2025Learning to Help in Multi-Class Settingscited by 2
  11. 2026Optimized Deferral for Imbalanced Settingscited by 2
  12. 2026Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Predictioncited by 2
  13. 2026DeferredSeg:A Multi-Expert Deferral Framework for Medical Image Segmentationcited by 1
  14. 2026Online Learning-to-Defer with Varying Expertscited by 1
  15. 2025A Greedy PDE Router for Blending Neural Operators and Classical Methodscited by 0
  16. 2026Principled Algorithms for Optimizing Generalized Metrics in Multi-Label Learningcited by 0
  17. 2026When Should a Satellite Estimate Be Changed? Stress-Testing Neural Corrections for Evapotranspirationcited by 0