# AgentMD: Empowering Language Agents for Risk Prediction with Large-Scale Clinical Tool Learning

arXiv: https://arxiv.org/abs/2402.13225 · HTML: https://mlsanity.com/paper/2402.13225/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 6 of its 7 references that are in the map.
- First posted on arXiv 20 Feb 2024.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 6 of 6)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning](https://mlsanity.com/paper/2311.10537/) (2023)
- [Large Language Models as Tool Makers](https://mlsanity.com/paper/2305.17126/) (2023)
- [CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets](https://mlsanity.com/paper/2309.17428/) (2023)
- [Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health](https://mlsanity.com/paper/2306.10070/) (2023)

## Built on by (most cited first, 30 of 33)

- [Tool Learning with Large Language Models: A Survey](https://mlsanity.com/paper/2405.17935/) (2024)
- [MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making](https://mlsanity.com/paper/2404.15155/) (2024)
- [Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents](https://mlsanity.com/paper/2503.24047/) (2025)
- [MedCalc-Bench: Evaluating Large Language Models for Medical Calculations](https://mlsanity.com/paper/2406.12036/) (2024)
- [LLM Agents Making Agent Tools](https://mlsanity.com/paper/2502.11705/) (2025)
- [A Survey on the Optimization of Large Language Model-based Agents](https://mlsanity.com/paper/2503.12434/) (2025)
- [A Survey of Large Language Model Empowered Agents for Recommendation and Search: Towards Next-Generation Information Retrieval](https://mlsanity.com/paper/2503.05659/) (2025)
- [MedAgentGym: A Scalable Agentic Training Environment for Code-Centric Reasoning in Biomedical Data Science](https://mlsanity.com/paper/2506.04405/) (2025)
- [ReflecTool: Towards Reflection-Aware Tool-Augmented Clinical Agents](https://mlsanity.com/paper/2410.17657/) (2024)
- [Cascaded Language Models for Cost-effective Human-AI Decision-Making](https://mlsanity.com/paper/2506.11887/) (2025)
- [Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physics](https://mlsanity.com/paper/2510.09901/) (2025)
- [How Well Can Modern LLMs Act as Agent Cores in Radiology Environments?](https://mlsanity.com/paper/2412.09529/) (2024)
- [A Perspective for Adapting Generalist AI to Specialized Medical AI Applications and Their Challenges](https://mlsanity.com/paper/2411.00024/) (2024)
- [Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safety](https://mlsanity.com/paper/2506.12482/) (2025)
- [JoyAgent-JDGenie: Technical Report on the GAIA](https://mlsanity.com/paper/2510.00510/) (2025)
- [CXRAgent: Director-Orchestrated Multi-Stage Reasoning for Chest X-Ray Interpretation](https://mlsanity.com/paper/2510.21324/) (2025)
- [RareAgents: Autonomous Multi-disciplinary Team for Rare Disease Diagnosis and Treatment](https://mlsanity.com/paper/2412.12475/) (2024)
- [SepsisCalc: Integrating Clinical Calculators into Early Sepsis Prediction via Dynamic Temporal Graph Construction](https://mlsanity.com/paper/2501.00190/) (2024)
- [MRAG: Benchmarking Retrieval-Augmented Generation for Bio-medicine](https://mlsanity.com/paper/2601.16503/) (2026)
- [RiskAgent: Synergizing Language Models with Validated Tools for Evidence-Based Risk Prediction](https://mlsanity.com/paper/2503.03802/) (2025)
- [AgentRx: A Benchmark Study of LLM Agents for Multimodal Clinical Prediction Tasks](https://mlsanity.com/paper/2605.10286/) (2026)
- [DynamiCare: A Dynamic Multi-Agent Framework for Interactive and Open-Ended Medical Decision-Making](https://mlsanity.com/paper/2507.02616/) (2025)
- [STAR : Bridging Statistical and Agentic Reasoning for Large Model Performance Prediction](https://mlsanity.com/paper/2602.12143/) (2026)
- [Agentic AI for Scientific Reasoning in Autonomous Quantum Sensing Experiments](https://mlsanity.com/paper/2607.25145/) (2026)
- [Humans and Large Language Models in Clinical Decision Support: A Study with Medical Calculators](https://mlsanity.com/paper/2411.05897/) (2024)
- [Pursuing Best Industrial Practices for Retrieval-Augmented Generation in the Medical Domain](https://mlsanity.com/paper/2602.03368/) (2026)
- [Agentic AI in Healthcare & Medicine: A Seven-Dimensional Taxonomy for Empirical Evaluation of LLM-based Agents](https://mlsanity.com/paper/2602.04813/) (2026)
- [Which Tool Response Should I Trust? Tool-Expertise-Aware Chest X-ray Agent with Multimodal Agentic Learning](https://mlsanity.com/paper/2602.21517/) (2026)
- [EyeAgent: An Agentic AI System for Multimodal Clinical Decision Support in Ophthalmology](https://mlsanity.com/paper/2511.09394/) (2025)
- [Process-Supervised Multi-Agent Reinforcement Learning for Reliable Clinical Reasoning](https://mlsanity.com/paper/2602.14160/) (2026)

## Abstract (arXiv)

Clinical calculators play a vital role in healthcare by offering accurate evidence-based predictions for various purposes such as prognosis. Nevertheless, their widespread utilization is frequently hindered by usability challenges, poor dissemination, and restricted functionality. Augmenting large language models with extensive collections of clinical calculators presents an opportunity to overcome these obstacles and improve workflow efficiency, but the scalability of the manual curation process poses a significant challenge. In response, we introduce AgentMD, a novel language agent capable of curating and applying clinical calculators across various clinical contexts. Using the published literature, AgentMD has automatically curated a collection of 2,164 diverse clinical calculators with executable functions and structured documentation, collectively named RiskCalcs. Manual evaluations show that RiskCalcs tools achieve an accuracy of over 80% on three quality metrics. At inference time, AgentMD can automatically select and apply the relevant RiskCalcs tools given any patient description. On the newly established RiskQA benchmark, AgentMD significantly outperforms chain-of-thought prompting with GPT-4 (87.7% vs. 40.9% in accuracy). Additionally, we also applied AgentMD to real-world clinical notes for analyzing both population-level and risk-level patient characteristics. In summary, our study illustrates the utility of language agents augmented with clinical calculators for healthcare analytics and patient care.