mlsanityData to 14 Oct 2026

Semantic Uncertainty-Guided Orchestration in Hierarchical Multi-Agent Systems

arXiv 2608.14707 ↗First posted 11 Aug 2026cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

As large language model (LLM)-based multi-agent systems become increasingly capable, coordinating agents under uncertainty becomes a fundamental challenge. Existing orchestration strategies typically rely on fixed interaction patterns and often lack mechanisms for assessing the reliability of intermediate reasoning steps, allowing errors and hallucinations to propagate through the system. This paper introduces a semantic-uncertainty-guided orchestration approach, HASSUM as a general framework for uncertainty-aware coordination in multi-agent systems. The method estimates uncertainty using semantic entropy and semantic density, which measure trust at the level of answer semantics rather than output probabilities. These signals enable adaptive orchestration decisions, including output verification, selective reprompting, additional deliberation, and confidence-aware response selection. Because the approach operates independently of any particular agent architecture, it can be integrated into a broad range of hierarchical and collaborative multi-agent systems. The evaluations demonstrate an implementation within a hierarchical agent framework and evaluate it on StrategyQA, JailbreakBench, and TruthfulQA benchmarks. Across tasks that require complex reasoning and are prone to ambiguity or hallucinations, uncertainty-guided orchestration yields more reliable outcomes than uncertainty-unaware coordination. Semantic entropy and semantic density in tandem outperformed either metric alone. Ablations testing different thresholds and model sizes demonstrated that both influence the effectiveness of semantic metrics. The results suggest that semantic uncertainty is a practical and general-purpose signal for improving robustness and trustworthiness in agentic AI systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 21 of its 25 references that are in the map. First posted on arXiv 11 Aug 2026.

Builds on

21 of 25 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Guided Collaboration in Heterogeneous LLM-Based Multi-Agent Systems via Entropy-Based Understanding Assessment and Experience Retrieval14 Feb
  2. 2026When Does Multi-Agent Collaboration Help? An Entropy Perspective4 Feb
  3. 2025UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Making20 Jun
  4. 2025AgentGroupChat-V2: Divide-and-Conquer Is What LLM-Based Multi-Agent System Need18 Jun
  5. 2025HASHIRU: Hierarchical Agent System for Hybrid Intelligent Resource Utilization1 Jun
  6. 2024SAUP: Situation Awareness Uncertainty Propagation on LLM Agent2 Dec
  7. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar

Other references · 14

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  4. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  5. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  6. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  7. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
  8. 2020MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformerscited by 245
  9. 2025Debate Only When Necessary: Adaptive Multiagent Collaboration for Efficient LLM Reasoningcited by 27
  10. 2024DebUnc: Improving Large Language Model Agent Communication With Uncertainty Metricscited by 14
  11. 2025Towards Agents That Know When They Don't Know: Uncertainty as a Control Signal for Structured Reasoningcited by 10
  12. 2026Rethinking the Role of Entropy in Optimizing Tool-Use Behaviors for Large Language Model Agentscited by 5
2 more
  1. 2026DiscoUQ: Structured Disagreement Analysis for Uncertainty Quantification in LLM Agent Ensemblescited by 4
  2. 2026Agentic AI-based Framework for Mitigating Premature Diagnostic Handoff and Silent Hallucination in Healthcare Applicationscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Multi-Agent Coordination Adaptation via Structure-Guided Orchestration
  2. 2026CoE: Collaborative Entropy for Uncertainty Quantification in Agentic Multi-LLM Systems
  3. 2026Recognize Your Orchestrator: An Entropy Dynamics Perspective for LLM Multi-Agent Systems
  4. 2026Every Response Counts: Quantifying Uncertainty of LLM-based Multi-Agent Systems through Tensor Decomposition
  5. 2026Silo-Bench: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems
  6. 2025Reasoning-Aware Prompt Orchestration: A Foundation Model for Multi-Agent Language Model Coordination
  7. 2024Towards Uncertainty-Aware Language Agent
  8. 2023LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models