mlsanityData to 14 Oct 2026

SAGE: Answer-Conditioned Uncertainty Targets for Verbal Uncertainty Alignment

arXiv 2606.11512 ↗First posted 9 Jun 2026cs.CLcited by 3

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Large language models increasingly express uncertainty through natural-language statements, yet these expressions often fail to reflect the model's sampled behavior. We study verbal uncertainty alignment as a distributional calibration problem: the appropriate uncertainty target for a prompt should be estimated from repeated model outputs rather than from an isolated response. However, group rollouts alone are insufficient, since the resulting target must provide a useful training signal. Existing targets only partially satisfy this requirement. We propose SAGE, Semantic-Answer Guided Entropy, a group-level uncertainty target that constructs an answer-conditioned uncertainty geometry over sampled responses. SAGE preserves categorical, numeric, and symbolic answer distinctions while maintaining a smooth and scale-preserving calibration signal. We further apply this target through Group-Uncertainty Preference Optimization, or GUPO, an uncertainty-channel training framework that supervises verbal uncertainty expressions rather than the full response. Experiments across factual, mathematical, and multiple-choice reasoning tasks show improved uncertainty ranking, lower calibration error, and reduced overconfidence.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 28 of its 42 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

28 of 42 references.

References · 28

  1. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  2. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  3. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  4. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  5. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  6. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  7. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 162
  8. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 159
  9. 2023LM-Polygraph: Uncertainty Estimation for Language Modelscited by 131
  10. 2024Language Model Cascades: Token-level uncertainty and beyondcited by 121
  11. 2024LUQ: Long-text Uncertainty Quantification for LLMscited by 81
  12. 2025The Obvious Invisible Threat: LLM-Powered GUI Agents' Vulnerability to Fine-Print Injectionscited by 59
16 more
  1. 2024CLEAR: Towards Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation for Large Language Model Applicationscited by 33
  2. 2025AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative Multi-Agent Question Answeringcited by 32
  3. 2025Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarkscited by 28
  4. 2024Finetuning Language Models to Emit Linguistic Expressions of Uncertaintycited by 26
  5. 2025AutoData: A Multi-Agent System for Open Web Data Collectioncited by 25
  6. 2024Deliberate Reasoning in Language Models as Structure-Aware Planning with an Accurate World Modelcited by 23
  7. 2025LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form Generationscited by 17
  8. 2025MAPRO: Recasting Multi-Agent Prompt Optimization as Maximum a Posteriori Inferencecited by 17
  9. 2026EvolveRouter: Co-Evolving Routing and Prompt for Multi-Agent Question Answeringcited by 15
  10. 2024ChefFusion: Multimodal Foundation Model Integrating Recipe and Food Image Generationcited by 11
  11. 2024Graph-based Uncertainty Metrics for Long-form Language Model Outputscited by 11
  12. 2025NG-Router: Graph-Supervised Multi-Agent Collaboration for Nutrition Question Answeringcited by 10
  13. 2025Verbalized Confidence Triggers Self-Verification: Emergent Behavior Without Explicit Reasoning Supervisioncited by 9
  14. 2026Drift-Bench: Diagnosing Cooperative Breakdowns in LLM Agents under Input Faults via Multi-Turn Interactioncited by 9
  15. 2026GLEN-Bench: A Graph-Language based Benchmark for Nutritional Healthcited by 7
  16. 2026Why Semantic Entropy Fails: Geometry-Aware and Calibrated Uncertainty for Policy Optimizationcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Optimizing What Policies Learn From: Recoverability-aware Rollout Intervention Learningcited by 4
  2. 2026On the Pitfalls of Verbalized Confidence Priors for Calibrating Large Reasoning Modelscited by 0
  3. 2026CLEAT: Learning Value-Guided Clarification for Interactive Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ORCE: Order-Aware Alignment of Verbalized Confidence in Large Language Models
  2. 2026From Entropy to Calibrated Uncertainty: Training Language Models to Reason About Uncertainty
  3. 2025Human-Alignment and Calibration of Inference-Time Uncertainty in Large Language Models
  4. 2026Semantic Uncertainty Quantification Needs Factual Equivalence
  5. 2026Sensitivity Uncertainty Alignment in Large Language Models ↗
  6. 2026Uncertainty-Aware Abstention in Large Language Models with Provable Alignment Guarantees
  7. 2024UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models
  8. 2025SEED-GRPO: Semantic Entropy Enhanced GRPO for Uncertainty-Aware Policy Optimization