# Advancing Healthcare Automation: Multi-Agent System for Medical Necessity Justification

arXiv: https://arxiv.org/abs/2404.17977 · HTML: https://mlsanity.com/paper/2404.17977/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 27 Apr 2024.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 16 of 16)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback](https://mlsanity.com/paper/2305.10142/) (2023)
- [A Survey of Large Language Models in Medicine: Progress, Application, and Challenge](https://mlsanity.com/paper/2311.05112/) (2023)
- [Enhancing Clinical Efficiency through LLM: Discharge Note Generation for Cardiac Patients](https://mlsanity.com/paper/2404.05144/) (2024)
- [Hey AI, Can You Solve Complex Tasks by Talking to Agents?](https://mlsanity.com/paper/2110.08542/) (2021)
- [Multimodal Fusion of EHR in Structures and Semantics: Integrating Clinical Records and Notes with Hypergraph and LLM](https://mlsanity.com/paper/2403.08818/) (2024)
- [Retrieving Evidence from EHRs with LLMs: Possibilities and Challenges](https://mlsanity.com/paper/2309.04550/) (2023)

## Built on by (most cited first, 10 of 10)

- [KG4Diagnosis: A Hierarchical Multi-Agent LLM Framework with Knowledge Graph Enhancement for Medical Diagnosis](https://mlsanity.com/paper/2412.16833/) (2024)
- [MDTeamGPT: A Self-Evolving LLM-based Multi-Agent Framework for Multi-Disciplinary Team Medical Consultation](https://mlsanity.com/paper/2503.13856/) (2025)
- [HealthAdminBench: Evaluating Computer-Use Agents on Healthcare Administration Tasks](https://mlsanity.com/paper/2604.09937/) (2026)
- [Agentic AI Systems Applied to tasks in Financial Services: Modeling and model risk management crews](https://mlsanity.com/paper/2502.05439/) (2025)
- [Mediator-Guided Multi-Agent Collaboration among Open-Source Models for Medical Decision-Making](https://mlsanity.com/paper/2508.05996/) (2025)
- [Evaluating Theory of Mind and Internal Beliefs in LLM-Based Multi-Agent Systems](https://mlsanity.com/paper/2603.00142/) (2026)
- [Generative Active Testing: Efficient LLM Evaluation via Proxy Task Adaptation](https://mlsanity.com/paper/2603.19264/) (2026)
- [MedCoAct: Confidence-Aware Multi-Agent Collaboration for Complete Clinical Decision](https://mlsanity.com/paper/2510.10461/) (2025)
- [Agentic AI in Healthcare & Medicine: A Seven-Dimensional Taxonomy for Empirical Evaluation of LLM-based Agents](https://mlsanity.com/paper/2602.04813/) (2026)
- [Digital Twin Ecosystem for Oncology Clinical Operations](https://mlsanity.com/paper/2409.17650/) (2024)

## Abstract (arXiv)

Prior Authorization delivers safe, appropriate, and cost-effective care that is medically justified with evidence-based guidelines. However, the process often requires labor-intensive manual comparisons between patient medical records and clinical guidelines, that is both repetitive and time-consuming. Recent developments in Large Language Models (LLMs) have shown potential in addressing complex medical NLP tasks with minimal supervision. This paper explores the application of Multi-Agent System (MAS) that utilize specialized LLM agents to automate Prior Authorization task by breaking them down into simpler and manageable sub-tasks. Our study systematically investigates the effects of various prompting strategies on these agents and benchmarks the performance of different LLMs. We demonstrate that GPT-4 achieves an accuracy of 86.2% in predicting checklist item-level judgments with evidence, and 95.6% in determining overall checklist judgment. Additionally, we explore how these agents can contribute to explainability of steps taken in the process, thereby enhancing trust and transparency in the system.