mlsanityData to 7 Oct 2026

Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safety

arXiv 2506.12482 ↗First posted 14 Jun 2025cs.AIcited by 7

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

Large language models (LLMs) deployed as agents introduce significant safety risks in clinical settings due to their potential for error and single points of failure. We introduce Tiered Agentic Oversight (TAO), a hierarchical multi-agent system that enhances AI safety through layered, automated supervision. Inspired by clinical hierarchies (e.g., nurse-physician-specialist) in hospital, TAO routes tasks to specialized agents based on complexity, creating a robust safety framework through automated inter- and intra-tier communication and role-playing. Crucially, this hierarchical structure functions as an effective error-correction mechanism, absorbing up to 24% of individual agent errors before they can compound. Our experiments reveal TAO outperforms single-agent and other multi-agent systems on 4 out of 5 healthcare safety benchmarks, with up to an 8.2% improvement. Ablation studies confirm key design principles of the system: (i) its adaptive architecture is over 3% safer than static, single-tier configurations, and (ii) its lower tiers are indispensable, as their removal causes the most significant degradation in overall safety. Finally, we validated the system's synergy with human doctors in a user study where a physician, acting as the highest tier agent, provided corrective feedback that improved medical triage accuracy from 40% to 60%. Project Page: https://tiered-agentic-oversight.github.io/

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 81 of its 118 references that are in the map. First posted on arXiv 14 Jun 2025.

Builds on

81 of 118 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025BehaviorSFT: Behavioral Token Conditioning for Clinical Agents Across the Proactivity Spectrum27 May
  2. 2024Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents5 May
  3. 2024AgentMD: Empowering Language Agents for Risk Prediction with Large-Scale Clinical Tool Learning20 Feb

Other references · 78

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  6. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  7. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  8. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  9. 2016Concrete Problems in AI Safetycited by 1,582
  10. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  11. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  12. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
38 more
  1. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  2. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  3. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  4. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
  5. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  6. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  7. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  8. 2023Cognitive Architectures for Language Agentscited by 422
  9. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  10. 2018AI safety via debatecited by 384
  11. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 375
  12. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 372
  13. 2025Accelerating scientific discovery with Co-Scientistcited by 353
  14. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 348
  15. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 319
  16. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 311
  17. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
  18. 2021Unsolved Problems in ML Safetycited by 291
  19. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 255
  20. 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 244
  21. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 235
  22. 2024Capabilities of Gemini Models in Medicinecited by 231
  23. 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 212
  24. 2023Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedbackcited by 202
  25. 2025Survey on Evaluation of LLM-based Agentscited by 190
  26. 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 189
  27. 2024MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Makingcited by 162
  28. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 156
  29. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 135
  30. 2024MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolutioncited by 132
  31. 2023ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Modelscited by 116
  32. 2023GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Informationcited by 115
  33. 2023Towards Accurate Differential Diagnosis with Large Language Modelscited by 115
  34. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
  35. 2024Improving Multi-Agent Debate with Sparse Communication Topologycited by 103
  36. 2024On scalable oversight with weak LLMs judging strong LLMscited by 94
  37. 2024WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Taskscited by 81
  38. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 80

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2025Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoningcited by 27
  2. 2025LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarioscited by 25
  3. 2026Sovereign Agentic Loops: Decoupling AI Reasoning from Execution in Real-World Systemscited by 5
  4. 2026From Black-Box Confidence to Measurable Trust in Clinical AI: A Framework for Evidence, Supervision, and Staged Autonomycited by 2
  5. 2026TRACE: A Metrologically-Grounded Engineering Framework for Trustworthy Agentic AI Systems in Operationally Critical Domainscited by 1
  6. 2026The Horizon Gap: Planning, Memory, Execution, Training, and Evaluation for Long-Horizon LLM Agentscited by 1
  7. 2026Trust Is Not a Score: Runtime Assurance Contracts for High-Risk AI Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Improving the Safety and Trustworthiness of Medical AI via Multi-Agent Evaluation Loops
  2. 2026ETHOS: Towards a Modular Ethics Framework for Clinical Multi-Agent Systems
  3. 2024Safeguarding AI Agents: Developing and Analyzing Safety Architectures
  4. 2026A Safety-Aware Role-Orchestrated Multi-Agent LLM Framework for Behavioral Health Communication Simulation
  5. 2026The Doctor Will (Still) See You Now: On the Structural Limits of Agentic AI in Healthcare
  6. 2025AgenticEval: Toward Agentic and Self-Evolving Safety Evaluation of Large Language Models
  7. 2026Agentic AI in Healthcare & Medicine: A Seven-Dimensional Taxonomy for Empirical Evaluation of LLM-based Agents
  8. 2026SafePro: Evaluating the Safety of Professional-Level AI Agents