mlsanityData to 7 Oct 2026

Evolving Medical Imaging Agents via Experience-driven Self-skill Discovery

arXiv 2603.05860 ↗First posted 6 Mar 2026cs.AI cs.CVcited by 10

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

Clinical image interpretation is inherently multi-step and tool-centric: clinicians iteratively combine visual evidence with patient context, quantify findings, and refine their decisions through a sequence of specialized procedures. While LLM-based agents promise to orchestrate such heterogeneous medical tools, existing systems treat tool sets and invocation strategies as static after deployment. This design is brittle under real-world domain shifts, across tasks, and evolving diagnostic requirements, where predefined tool chains frequently degrade and demand costly manual re-design. We propose MACRO, a self-evolving, experience-augmented medical agent that shifts from static tool composition to experience-driven tool discovery. From verified execution trajectories, the agent autonomously identifies recurring effective multi-step tool sequences, synthesizes them into reusable composite tools, and registers these as new high-level primitives that continuously expand its behavioral repertoire. A lightweight image-feature memory grounds tool selection in a visual-clinical context, while a GRPO-like training loop reinforces reliable invocation of discovered composites, enabling closed-loop self-improvement with minimal supervision. Extensive experiments across diverse medical imaging datasets and tasks demonstrate that autonomous composite tool discovery consistently improves multi-step orchestration accuracy and cross-domain generalization over strong baselines and recent state-of-the-art agentic methods, bridging the gap between brittle static tool use and adaptive, context-aware clinical AI assistance. Code will be available upon acceptance.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 39 of its 54 references that are in the map. First posted on arXiv 6 Mar 2026.

Builds on

39 of 54 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Medical AI Consensus: A Multi-Agent Framework for Radiology Report Generation and Evaluation22 Sep
  2. 2025Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactions28 Mar
  3. 2024Medchain: Bridging the Gap Between LLM Agents and Clinical Practice with Interactive Sequence2 Dec
  4. 2024MMedAgent: Learning to Use Medical Tools with Multi-modal Agent2 Jul
  5. 2024Beyond Direct Diagnosis: LLM-based Multi-Specialist Agent Consultation for Automatic Diagnosis29 Jan

Other references · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  7. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  8. 2023A Survey of Large Language Modelscited by 2,537
  9. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  10. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  11. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  12. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
22 more
  1. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  2. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  3. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 636
  4. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
  5. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  6. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 311
  7. 2023Towards Generalist Biomedical AIcited by 286
  8. 2022EfficientFormer: Vision Transformers at MobileNet Speedcited by 240
  9. 2024MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Makingcited by 162
  10. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecyclecited by 129
  11. 2026MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agentscited by 129
  12. 2024MedMamba: Vision Mamba for Medical Image Classificationcited by 86
  13. 2024Convolutional Kolmogorov-Arnold Networkscited by 76
  14. 2024EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Recordscited by 75
  15. 2025MedRAX: Medical Reasoning Agent for Chest X-raycited by 68
  16. 2024CoD, Towards an Interpretable Medical Agent using Chain of Diagnosiscited by 43
  17. 2022REFUGE2 Challenge: A Treasure Trove for Multi-Dimension Analysis and Evaluation in Glaucoma Screeningcited by 37
  18. 2023Large Language Models Streamline Automated Machine Learning for Clinical Studiescited by 24
  19. 2025Beyond ReAct: A Planner-Centric Framework for Complex Tool-Augmented LLM Reasoningcited by 16
  20. 2024Almanac Copilot: Towards Autonomous Electronic Health Record Navigationcited by 9
  21. 2025RAM-W600: A Multi-Task Wrist Dataset and Benchmark for Rheumatoid Arthritiscited by 5
  22. 2025Retrieval-Augmented Framework for LLM-Based Clinical Decision Supportcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 10.

10 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards Autonomous and Auditable Medical Imaging Model Development
  2. 2026Evo-MedAgent: Beyond One-Shot Diagnosis with Agents That Remember, Reflect, and Improve
  3. 2025Simple Agents Outperform Experts in Biomedical Imaging Workflow Optimization
  4. 2026Can Coding Agents Build Robust Baselines? A Skill-Based Approach for Automating the Medical Imaging Model-Development Pipeline
  5. 2025ReX-MLE: The Autonomous Agent Benchmark for Medical Imaging Challenges
  6. 2025M^3Builder: A Multi-Agent System for Automated Machine Learning in Medical Imaging
  7. 2026Route, Retrieve, Reflect, Repair: Self-Improving Agentic Framework for Visual Detection and Linguistic Reasoning in Medical Imaging
  8. 2026MIRA: Medical Image Reflection for Agentic Diagnosis