mlsanityData to 7 Oct 2026

Learning to Defer to a Population: A Meta-Learning Approach

arXiv 2403.02683 ↗First posted 5 Mar 2024cs.LG stat.MLcited by 26

Not on the map. Why?

Abstract (arXiv)

The learning to defer (L2D) framework allows autonomous systems to be safe and robust by allocating difficult decisions to a human expert. All existing work on L2D assumes that each expert is well-identified, and if any expert were to change, the system should be re-trained. In this work, we alleviate this constraint, formulating an L2D system that can cope with never-before-seen experts at test-time. We accomplish this by using meta-learning, considering both optimization- and model-based variants. Given a small context set to characterize the currently available expert, our framework can quickly adapt its deferral policy. For the model-based approach, we employ an attention mechanism that is able to look for points in the context set that are similar to a given test point, leading to an even more precise assessment of the expert's abilities. In the experiments, we validate our methods on image recognition, traffic sign detection, and skin lesion diagnosis benchmarks.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 36 of its 62 references that are in the map. First posted on arXiv 5 Mar 2024.

Builds on

36 of 62 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2016Matching Networks for One Shot Learningcited by 2,246
  4. 2017Deep Setscited by 1,394
  5. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  6. 2018Conditional Neural Processescited by 421
  7. 2018How to train your MAMLcited by 306
  8. 2019Attentive Neural Processescited by 280
  9. 2020Consistent Estimators for Learning to Defer to an Expertcited by 262
  10. 2017Predict Responsibly: Improving Fairness and Accuracy by Learning to Defercited by 228
  11. 2018To Trust Or Not To Trust A Classifiercited by 205
  12. 2019Convolutional Conditional Neural Processescited by 139
24 more
  1. 2020Learning to Complement Humanscited by 123
  2. 2019The Algorithmic Automation Problem: Prediction, Triage, and Human Effortcited by 101
  3. 2020Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processescited by 74
  4. 2021Differentiable Learning Under Triagecited by 74
  5. 2021Towards Unbiased and Accurate Deferral to Multiple Expertscited by 71
  6. 2022Calibrated Learning to Defer with One-vs-All Classifierscited by 68
  7. 2023Who Should Predict? Exact Algorithms For Learning to Defer to Humanscited by 68
  8. 2022Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensemblescited by 59
  9. 2020Classification with Rejection Based on Cost-sensitive Classificationcited by 55
  10. 2021Hybrid Intelligencecited by 55
  11. 2020TaskNorm: Rethinking Batch Normalization for Meta-Learningcited by 50
  12. 2022Sample Efficient Learning of Predictors that Complement Humanscited by 46
  13. 2019Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networkscited by 45
  14. 2019On the Calibration of Multiclass Classification with Rejectioncited by 41
  15. 2022Forming Effective Human-AI Teams: Building Machine Learning Models that Complement the Capabilities of Multiple Expertscited by 40
  16. 2023Principled Approaches for Learning to Defer with Multiple Expertscited by 38
  17. 2020Regression with reject option and application to kNNcited by 30
  18. 2023In Defense of Softmax Parametrization for Calibrated and Consistent Learning to Defercited by 26
  19. 2023Learning to Defer with Limited Expert Predictionscited by 25
  20. 2021Preferential Mixture-of-Experts: Interpretable Models that Rely on Human Expertise as much as Possiblecited by 19
  21. 2021Group Equivariant Conditional Neural Processescited by 16
  22. 2022Human-AI Collaboration in Decision-Making: Beyond Learning to Defercited by 16
  23. 2021Incorporating Uncertainty in Learning to Defer Algorithms for Safe Computer-Aided Diagnosiscited by 15
  24. 2023Exploiting Inferential Structure in Neural Processescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 11; 2026: 10.

26 papers.

Sort:

By area:

  1. 2025Universal Model Routing for Efficient LLM Inferencecited by 80
  2. 2024Regression with Multi-Expert Deferralcited by 29
  3. 2024Realizable H-Consistent and Bayes-Consistent Loss Functions for Learning to Defercited by 26
  4. 2024A Two-Stage Learning-to-Defer Approach for Multi-Task Learningcited by 16
  5. 2025Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees for Learning to Defercited by 16
  6. 2024Prune 'n Predict: Optimizing LLM Decision-making with Conformal Predictioncited by 15
  7. 2025Budgeted Multiple-Expert Deferralcited by 11
  8. 2025Why Ask One When You Can Ask k? Learning-to-Defer to the Top-k Expertscited by 6
  9. 2024A Unifying Post-Processing Framework for Multi-Objective Learn-to-Defer Problemscited by 5
  10. 2025Adversarial Robustness in One-Stage Learning-to-Defercited by 5
  11. 2025Identity-Free Deferral For Unseen Expertscited by 4
  12. 2025Cost-Aware Routing for Efficient Text-To-Image Generationcited by 4
14 more
  1. 2026Fatigue-Aware Learning to Defer via Constrained Optimisationcited by 3
  2. 2025Learning to Help in Multi-Class Settingscited by 2
  3. 2025No Need for Learning to Defer? A Training Free Deferral Framework to Multiple Experts through Conformal Predictioncited by 2
  4. 2026L2CU: Learning to Complement Unseen Userscited by 2
  5. 2026When More Experts Hurt: Underfitting in Multi-Expert Learning to Defercited by 2
  6. 2026Optimized Deferral for Imbalanced Settingscited by 2
  7. 2025From Deferral to Learning: Online In-Context Knowledge Distillation for LLM Cascadescited by 1
  8. 2026Consistent Learning-to-Defer with Expert-Conditional Advicecited by 1
  9. 2025Learning To Defer To A Population With Limited Demonstrationscited by 0
  10. 2026MPD2-Router: Mask-aware Multi-expert Prior-regularized Dual-head Deferral Router in Glaucoma Screening and Diagnosiscited by 0
  11. 2026Online Learning-to-Defer with Varying Expertscited by 0
  12. 2026Human-AI Teaming Through the Lens of Calibrationcited by 0
  13. 2026From Few-Shot Segmentation to Clinician-in-the-Loop Medical Image Analysiscited by 0
  14. 2026RACER: Role-Aligned Competence Estimation for Human-AI Routingcited by 0