mlsanityData to 7 Oct 2026

Runtime Uncertainty Monitoring for LLM-Based Multi-Agent Systems Using Bayesian Networks

arXiv 2607.25877 ↗First posted 28 Jul 2026cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

This paper investigates how multi-agent systems (MAS)-based on large language models (LLMs) can support actuarial risk modelling, with a particular focus on uncertainty quantification. Actuarial workflows represent a high-stakes decision-support setting where unreliable outputs may lead to incorrect risk assessment, unfair pricing, and regulatory non-compliance. To address uncertainty introduced by the probabilistic nature of LLMs and dependencies between agents, a multi-agent framework is proposed in which specialised agents perform data preparation, modelling, review, and explanation tasks under a central hub. The main contribution is a novel approach to uncertainty propagation using token-level log-probabilities and a Bayesian Network. Importantly, log probabilities are not treated as direct probabilities of correctness or task success. Instead, length-normalised log-probability summaries are transformed into calibrated task-level confidence estimates before incorporation into the Bayesian Network. Results show that the framework reproduces baseline actuarial performance while providing additional insight into workflow stability and runtime uncertainty propagation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 14 of its 24 references that are in the map. First posted on arXiv 28 Jul 2026.

Builds on

14 of 24 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Evaluation and Benchmarking of LLM Agents: A Survey29 Jul
  2. 2025UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Making20 Jun
  3. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges15 May
  4. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMs10 Jan
  5. 2024LLM-based Multi-Agent Systems: Techniques and Business Perspectives21 Nov
  6. 2023A Survey on Large Language Model based Autonomous Agents22 Aug

Other references · 8

  1. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  2. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  3. 2025SentinelAgent: Graph-based Anomaly Detection in Multi-Agent Systemscited by 22
  4. 2025Extracting Probabilistic Knowledge from Large Language Models for Bayesian Network Parameterizationcited by 15
  5. 2025Log Probability Tracking of LLM APIscited by 8
  6. 2024A Bayesian Approach to Harnessing the Power of LLMs in Authorship Attributioncited by 3
  7. 2025Advanced Applications of Generative AI in Actuarial Science: Case Studies Beyond ChatGPTcited by 3
  8. 2026Bayesian Uncertainty Propagation for Agentic RAG Pipelines: A Proof-of-Concept Study on Multi-Hop Question Answeringcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026PropUQ-MAS: Propagation-Aware Uncertainty Quantification for LLM Multi-Agent Systems
  2. 2026From Sequence to Structure: Relational Uncertainty Propagation for LLM Agents
  3. 2026Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunities
  4. 2025Risk Analysis Techniques for Governed LLM-based Multi-Agent Systems
  5. 2024SAUP: Situation Awareness Uncertainty Propagation on LLM Agent
  6. 2026Agentic Trading: When LLM Agents Meet Financial Markets
  7. 2026Can LLM Agents Be CFOs? Benchmarking Long-Horizon Resource Allocation in an Uncertain Enterprise Environment
  8. 2025Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agents