# HealthAdminBench: Evaluating Computer-Use Agents on Healthcare Administration Tasks

arXiv: https://arxiv.org/abs/2604.09937 · HTML: https://mlsanity.com/paper/2604.09937/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 14 of its 22 references that are in the map.
- First posted on arXiv 10 Apr 2026.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 14 of 14)

- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models](https://mlsanity.com/paper/2401.13919/) (2024)
- [HealthBench: Evaluating Large Language Models Towards Improved Human Health](https://mlsanity.com/paper/2505.08775/) (2025)
- [WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?](https://mlsanity.com/paper/2403.07718/) (2024)
- [Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration](https://mlsanity.com/paper/1802.08802/) (2018)
- [Med-HALT: Medical Domain Hallucination Test for Large Language Models](https://mlsanity.com/paper/2307.15343/) (2023)
- [MedAgentBench: A Realistic Virtual EHR Environment to Benchmark Medical LLM Agents](https://mlsanity.com/paper/2501.14654/) (2025)
- [REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites](https://mlsanity.com/paper/2504.11543/) (2025)
- [Web-Bench: A LLM Code Benchmark Based on Web Standards and Frameworks](https://mlsanity.com/paper/2505.07473/) (2025)
- [FHIR-AgentBench: Benchmarking LLM Agents for Realistic Interoperable EHR Question Answering](https://mlsanity.com/paper/2509.19319/) (2025)
- [Advancing Healthcare Automation: Multi-Agent System for Medical Necessity Justification](https://mlsanity.com/paper/2404.17977/) (2024)
- [Can GPT Improve the State of Prior Authorization via Guideline Based Automated Question Answering?](https://mlsanity.com/paper/2402.18419/) (2024)

## Built on by (most cited first, 13 of 13)

- [HealthBench Professional: Evaluating Large Language Models on Real Clinician Chats](https://mlsanity.com/paper/2604.27470/) (2026)
- [CHI-Bench: Can AI Agents Automate End-to-End, Long-Horizon, Policy-Rich Healthcare Workflows?](https://mlsanity.com/paper/2605.16679/) (2026)
- [HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents](https://mlsanity.com/paper/2606.31179/) (2026)
- [AutoMedBench: Towards Medical AutoResearch with Agentic AI Models](https://mlsanity.com/paper/2606.01961/) (2026)
- [MedCUA-Bench: A Screenshot-Only Benchmark for Clinical Computer-Use Agents](https://mlsanity.com/paper/2606.03203/) (2026)
- [How Benchmarks Mis-Score Computer-Use Agents](https://mlsanity.com/paper/2607.28367/) (2026)
- [MedCTA: A Benchmark for Clinical Tool Agents](https://mlsanity.com/paper/2606.11702/) (2026)
- [The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomy](https://mlsanity.com/paper/2607.11175/) (2026)
- [World Feedback for Clinical Agents: Diagnosing RL in FHIR Environments](https://mlsanity.com/paper/2607.01470/) (2026)
- [Synthetic Hospital: An Open, Verifiable, Physician-Validated Longitudinal EHR Benchmark](https://mlsanity.com/paper/2609.30027/) (2026)
- [CLAIRE: A Schema-Grounded Hybrid Workflow for Healthcare Administrative Form Completion](https://mlsanity.com/paper/2609.32787/) (2026)
- [MedImageOSWorld: Benchmarking GUI Agents for Medical Image Consoles](https://mlsanity.com/paper/2610.04800/) (2026)
- [MedHarnessBench: A Unified Benchmark for Healthcare Agent Evaluation](https://mlsanity.com/paper/2610.15714/) (2026)

## Abstract (arXiv)

Healthcare administration accounts for over 1 trillion in annual spending, making it a promising target for LLM-based computer-use agents (CUAs). While clinical applications of LLMs have received significant attention, no benchmark exists for evaluating CUAs on end-to-end administrative workflows. To address this gap, we introduce HealthAdminBench, a benchmark comprising four realistic GUI environments: an EHR, two payer portals, and a fax system, and 135 expert-defined tasks spanning three administrative task types: Prior Authorization, Appeals and Denials Management, and Durable Medical Equipment (DME) Order Processing. Each task is decomposed into fine-grained, verifiable subtasks, yielding 1,698 evaluation points. We evaluate seven agent configurations under multiple prompting and observation settings and find that, despite strong subtask performance, end-to-end reliability remains low: the best-performing agent (Claude Opus 4.6 CUA) achieves only 36.3 percent task success, while GPT-5.4 CUA attains the highest subtask success rate (82.8 percent). These results reveal a substantial gap between current agent capabilities and the demands of real-world administrative workflows. HealthAdminBench provides a rigorous foundation for evaluating progress toward safe and reliable automation of healthcare administrative workflows.