mlsanityData to 7 Oct 2026

Learning to Route LLMs with Confidence Tokens

arXiv 2410.13284 ↗First posted 17 Oct 2024cs.CL cs.AI cs.LGcited by 66

Not on the map. Why?

Cited by 29 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) have demonstrated impressive performance on several tasks and are increasingly deployed in real-world applications. However, especially in high-stakes settings, it becomes vital to know when the output of an LLM may be unreliable. Depending on whether an answer is trustworthy, a system can then choose to route the question to another expert, or otherwise fall back on a safe default behavior. In this work, we study the extent to which LLMs can reliably indicate confidence in their answers, and how this notion of confidence can translate into downstream accuracy gains. We propose Self-Reflection with Error-based Feedback (Self-REF), a lightweight training strategy to teach LLMs to express confidence in whether their answers are correct in a reliable manner. Self-REF introduces confidence tokens into the LLM, from which a confidence score can be extracted. Compared to conventional approaches such as verbalizing confidence and examining token probabilities, we demonstrate empirically that confidence tokens show significant improvements in downstream routing and rejection learning tasks.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Builds on 49 of its 68 references that are in the map. First posted on arXiv 17 Oct 2024.

Builds on

49 of 68 references.

References · 49

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  7. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  8. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  9. 2024Mixtral of Expertscited by 1,785
  10. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  11. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  12. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
37 more
  1. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  2. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  3. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  4. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  5. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  6. 2020Aligning AI With Shared Human Valuescited by 706
  7. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  8. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  9. 2024LoRA+: Efficient Low Rank Adaptation of Large Modelscited by 365
  10. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 354
  11. 2016Solving General Arithmetic Word Problemscited by 353
  12. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 286
  13. 2020Consistent Estimators for Learning to Defer to an Expertcited by 262
  14. 2020Selective Question Answering under Domain Shiftcited by 228
  15. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 222
  16. 2024Chain-of-Thought Reasoning Without Promptingcited by 208
  17. 2022Large Language Models with Controllable Working Memorycited by 169
  18. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 147
  19. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 140
  20. 2024To Believe or Not to Believe Your LLMcited by 120
  21. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 113
  22. 2022DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answeringcited by 93
  23. 2023Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Modelscited by 91
  24. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 85
  25. 2024Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approachcited by 73
  26. 2020Generating SOAP Notes from Doctor-Patient Conversations Using Modular Summarization Techniquescited by 64
  27. 2024Factual Confidence of LLMs: on Reliability and Robustness of Current Estimatorscited by 59
  28. 2022A Unifying Theory of Distance from Calibrationcited by 51
  29. 2021Knowing More About Questions Can Help: Improving Calibration in Question Answeringcited by 47
  30. 2023EcoAssistant: Using LLM Assistant More Affordably and Accuratelycited by 47
  31. 2024Multicalibration for Confidence Scoring in LLMscited by 44
  32. 2023Learning to Reject with a Fixed Predictor: Application to Decontextualizationcited by 32
  33. 2023Learning to Make Adherence-Aware Advicecited by 13
  34. 2023Ruffle&Riley: Towards the Automated Induction of Conversational Tutoring Systemscited by 12
  35. 2024Context-Aware Assistant Selection for Improved Inference Acceleration with Large Language Modelscited by 4
  36. 2024Recent Advances, Applications, and Open Challenges in Machine Learning for Health: Reflections from Research Roundtables at ML4H 2023 Symposiumcited by 3
  37. 2023When No-Rejection Learning is Consistent for Regression with Rejectioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 30; 2026: 36.

66 papers.

Sort:

By area:

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  2. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  3. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  4. 2025Deep Think with Confidencecited by 199
  5. 2025Reasoning Models Can Be Effective Without Thinkingcited by 182
  6. 2025Efficient Reasoning Models: A Surveycited by 93
  7. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  8. 2025Universal Model Routing for Efficient LLM Inferencecited by 80
  9. 2025Efficient Inference for Large Reasoning Models: A Surveycited by 58
  10. 2025Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resamplingcited by 32
  11. 2025When to Continue Thinking: Adaptive Thinking Mode Switching for Efficient Reasoningcited by 26
  12. 2025Shorten After You're Right: Lazy Length Penalties for Reasoning RLcited by 25
38 more
  1. 2025Adaptive LLM Routing under Budget Constraintscited by 21
  2. 2025RouterArena: An Open Platform for Comprehensive Comparison of LLM Routerscited by 19
  3. 2025TL;DR: Too Long, Do Re-weighting for Efficient LLM Reasoning Compressioncited by 16
  4. 2025xRouter: Training Cost-Aware LLMs Orchestration System via Reinforcement Learningcited by 16
  5. 2026Dynamic Model Routing and Cascading for Efficient LLM Inference: A Surveycited by 16
  6. 2025LAPO: Internalizing Reasoning Efficiency via Length-Adaptive Policy Optimizationcited by 11
  7. 2025Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoningcited by 9
  8. 2026Causal Evidence that Language Models use Confidence to Drive Behaviorcited by 9
  9. 2025Fast or Better? Balancing Accuracy and Cost in Retrieval-Augmented Generation with Flexible User Controlcited by 6
  10. 2025LLM Routing with Dueling Feedbackcited by 6
  11. 2026Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoringcited by 6
  12. 2026LLMs Should Express Uncertainty Explicitlycited by 6
  13. 2025IPR: Intelligent Prompt Routing with User-Controlled Quality-Cost Trade-offscited by 5
  14. 2025Generalized Correctness Models: Learning Calibrated and Model-Agnostic Correctness Predictors from Historical Patternscited by 5
  15. 2026Mitigating Overthinking in Large Reasoning Models via Difficulty-aware Reinforcement Learningcited by 5
  16. 2026Meissa: Multi-modal Medical Agentic Intelligencecited by 5
  17. 2026Calibration-Aware Policy Optimization for Reasoning LLMscited by 4
  18. 2025Think in Blocks: Adaptive Reasoning from Direct Response to Deep Reasoningcited by 3
  19. 2026TCAndon-Router: Adaptive Reasoning Router for Multi-Agent Collaborationcited by 3
  20. 2026Training Agents to Self-Report Misbehaviorcited by 3
  21. 2026Optimizing the Cost-Quality Tradeoff of Agentic Theorem Provers in Leancited by 3
  22. 2025Fine-Tuning and Prompt Engineering of LLMs, for the Creation of Multi-Agent AI for Addressing Sustainable Protein Production Challengescited by 2
  23. 2025Can Confidence Estimates Decide When Chain-of-Thought Is Necessary for LLMs?cited by 2
  24. 2026IntroLM: Introspective Language Models via Prefilling-Time Self-Evaluationcited by 2
  25. 2026Minimizing Mismatch Risk: A Prototype-Based Routing Framework for Zero-shot LLM-generated Text Detectioncited by 2
  26. 2026Inference Time Optimization with Confidence Dynamicscited by 2
  27. 2026SeqRoute: Global Budget-Aware Sequential LLM Routing via Offline Reinforcement Learningcited by 2
  28. 2026Uncertainty Is Not Enough: Value-of-Information Routing for Mixtures of LoRA Expertscited by 2
  29. 2025PIE: Performance Interval Estimation for Free-Form Generation Taskscited by 1
  30. 2025Fast Thinking for Large Language Modelscited by 1
  31. 2025Meta-Router: Bridging Gold-standard and Preference-based Evaluations in Large Language Model Routingcited by 1
  32. 2026Optimal Splitting of Language Models from Mixtures to Specialized Domainscited by 1
  33. 2026Reflective Prompt Tuning through Language Model Function-Callingcited by 1
  34. 2026FlyRoute: Self-Evolving Agent Profiling via Data Flywheel for Adaptive Task Routingcited by 1
  35. 2026Capability Self-Assessment in Large Language Modelscited by 1
  36. 2025A Greedy PDE Router for Blending Neural Operators and Classical Methodscited by 0
  37. 2025Gold-Switch: Training-Free Superposition of Slow- and Fast- Thinking LLMscited by 0
  38. 2026Fact-Checking with Large Language Models via Probabilistic Certainty and Consistencycited by 0

50 most cited of 66 All 66 (JSON)