# MedRAX: Medical Reasoning Agent for Chest X-ray

arXiv: https://arxiv.org/abs/2502.02673 · HTML: https://mlsanity.com/paper/2502.02673/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 4 Feb 2025.

## Builds on (most cited first, 21 of 21)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning](https://mlsanity.com/paper/1711.05225/) (2017)
- [SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answering](https://mlsanity.com/paper/2102.09542/) (2021)
- [AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments](https://mlsanity.com/paper/2405.07960/) (2024)
- [The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey](https://mlsanity.com/paper/2404.11584/) (2024)
- [MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making](https://mlsanity.com/paper/2404.15155/) (2024)
- [TorchXRayVision: A library of chest X-ray datasets and models](https://mlsanity.com/paper/2111.00595/) (2021)
- [MMedAgent: Learning to Use Medical Tools with Multi-modal Agent](https://mlsanity.com/paper/2407.02483/) (2024)
- [A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretation](https://mlsanity.com/paper/2401.12208/) (2024)
- [On the limits of cross-domain generalization in automated X-ray prediction](https://mlsanity.com/paper/2002.02497/) (2020)
- [Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V](https://mlsanity.com/paper/2310.19061/) (2023)
- [A Structure-Aware Relation Network for Thoracic Diseases Detection and Segmentation](https://mlsanity.com/paper/2104.10326/) (2021)
- [An In-depth Survey of Large Language Model-based Artificial Intelligence Agents](https://mlsanity.com/paper/2309.14365/) (2023)
- [MMAU: A Holistic Benchmark of Agent Capabilities Across Diverse Domains](https://mlsanity.com/paper/2407.18961/) (2024)
- [Rad-ReStruct: A Novel VQA Benchmark and Method for Structured Radiology Reporting](https://mlsanity.com/paper/2307.05766/) (2023)
- [M4CXR: Exploring Multi-task Potentials of Multi-modal Large Language Models for Chest X-ray Interpretation](https://mlsanity.com/paper/2408.16213/) (2024)
- [An Accurate and Explainable Deep Learning System Improves Interobserver Agreement in the Interpretation of Chest Radiograph](https://mlsanity.com/paper/2208.03545/) (2022)

## Built on by (most cited first, 30 of 70)

- [Large Language Model Agent: A Survey on Methodology, Applications and Challenges](https://mlsanity.com/paper/2503.21460/) (2025)
- [MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflow](https://mlsanity.com/paper/2503.18968/) (2025)
- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [El Agente: An Autonomous Agent for Quantum Chemistry](https://mlsanity.com/paper/2505.02484/) (2025)
- [BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM Model](https://mlsanity.com/paper/2505.23579/) (2025)
- [MedSAM3: Delving into Segment Anything with Medical Concepts](https://mlsanity.com/paper/2511.19046/) (2025)
- [Evolving Medical Imaging Agents via Experience-driven Self-skill Discovery](https://mlsanity.com/paper/2603.05860/) (2026)
- [Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications](https://mlsanity.com/paper/2508.00669/) (2025)
- [ADAgent: LLM Agent for Alzheimer's Disease Analysis with Collaborative Coordinator](https://mlsanity.com/paper/2506.11150/) (2025)
- [Emerging Cyber Attack Risks of Medical AI Agents](https://mlsanity.com/paper/2504.03759/) (2025)
- [RadAgents: Multimodal Agentic Reasoning for Chest X-ray Interpretation with Radiologist-like Workflows](https://mlsanity.com/paper/2509.20490/) (2025)
- [WSI-Agents: A Collaborative Multi-Agent System for Multi-Modal Whole Slide Image Analysis](https://mlsanity.com/paper/2507.14680/) (2025)
- [CXRAgent: Director-Orchestrated Multi-Stage Reasoning for Chest X-Ray Interpretation](https://mlsanity.com/paper/2510.21324/) (2025)
- [MTBBench: A Multimodal Sequential Clinical Decision-Making Benchmark in Oncology](https://mlsanity.com/paper/2511.20490/) (2025)
- [Meissa: Multi-modal Medical Agentic Intelligence](https://mlsanity.com/paper/2603.09018/) (2026)
- [Evo-MedAgent: Beyond One-Shot Diagnosis with Agents That Remember, Reflect, and Improve](https://mlsanity.com/paper/2604.14475/) (2026)
- [RadAgent: A tool-using AI agent for stepwise interpretation of chest computed tomography](https://mlsanity.com/paper/2604.15231/) (2026)
- [AURA: A Multi-Modal Medical Agent for Understanding, Reasoning & Annotation](https://mlsanity.com/paper/2507.16940/) (2025)
- [3DMedAgent: Unified Perception-to-Understanding for 3D Medical Analysis](https://mlsanity.com/paper/2602.18064/) (2026)
- [XrayClaw: Cooperative-Competitive Multi-Agent Alignment for Trustworthy Chest X-ray Diagnosis](https://mlsanity.com/paper/2604.02695/) (2026)
- [AT-CXR: Uncertainty-Aware Agentic Triage for Chest X-rays](https://mlsanity.com/paper/2508.19322/) (2025)
- [MEDVISTAGYM: A Scalable Training Environment for Thinking with Medical Images via Tool-Integrated Reinforcement Learning](https://mlsanity.com/paper/2601.07107/) (2026)
- [ECG-Agent: On-Device Tool-Calling Agent for ECG Multi-Turn Dialogue](https://mlsanity.com/paper/2601.20323/) (2026)
- [Medical thinking with multiple images](https://mlsanity.com/paper/2604.16506/) (2026)
- [Agentic Large Language Models for Training-Free Neuro-Radiological Image Analysis](https://mlsanity.com/paper/2604.16729/) (2026)
- [ClinSeekAgent: Automating Multimodal Evidence Seeking for Agentic Clinical Reasoning](https://mlsanity.com/paper/2605.20176/) (2026)
- [CX-Mind: A Pioneering Multimodal Large Language Model for Interleaved Reasoning in Chest X-ray via Curriculum-Guided Reinforcement Learning](https://mlsanity.com/paper/2508.03733/) (2025)
- [EndoCogniAgent: Closed-Loop Agentic Reasoning with Self-Consistency Validation for Endoscopic Diagnosis](https://mlsanity.com/paper/2508.07292/) (2025)
- [CXReasonAgent: Evidence-Grounded Diagnostic Reasoning Agent for Chest X-rays](https://mlsanity.com/paper/2602.23276/) (2026)
- [OralAgent: Integrating Reasoning, Tools, and Knowledge for Interactive Dental Image Analysis](https://mlsanity.com/paper/2605.27378/) (2026)

## Abstract (arXiv)

Chest X-rays (CXRs) play an integral role in driving critical decisions in disease management and patient care. While recent innovations have led to specialized models for various CXR interpretation tasks, these solutions often operate in isolation, limiting their practical utility in clinical practice. We present MedRAX, the first versatile AI agent that seamlessly integrates state-of-the-art CXR analysis tools and multimodal large language models into a unified framework. MedRAX dynamically leverages these models to address complex medical queries without requiring additional training. To rigorously evaluate its capabilities, we introduce ChestAgentBench, a comprehensive benchmark containing 2,500 complex medical queries across 7 diverse categories. Our experiments demonstrate that MedRAX achieves state-of-the-art performance compared to both open-source and proprietary models, representing a significant step toward the practical deployment of automated CXR interpretation systems. Data and code have been publicly available at https://github.com/bowang-lab/MedRAX