# From Clinical Intent to Clinical Model: Autonomous Coding-Agents for Clinician-driven AI Development

arXiv: https://arxiv.org/abs/2604.17110 · HTML: https://mlsanity.com/paper/2604.17110/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 61 references that are in the map.
- First posted on arXiv 18 Apr 2026.

## Builds on (most cited first, 30 of 34)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [Accelerating scientific discovery with Co-Scientist](https://mlsanity.com/paper/2502.18864/) (2025)
- [MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents](https://mlsanity.com/paper/2506.15841/) (2025)
- [Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging](https://mlsanity.com/paper/1909.12475/) (2019)
- [Towards End-to-End Automation of AI Research](https://mlsanity.com/paper/2606.15497/) (2026)
- [Reading Race: AI Recognises Patient's Racial Identity In Medical Images](https://mlsanity.com/paper/2107.10356/) (2021)
- [Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering](https://mlsanity.com/paper/2604.08224/) (2026)
- [A Review of Challenges and Opportunities in Machine Learning for Health](https://mlsanity.com/paper/1806.00388/) (2018)
- [AutoML to Date and Beyond: Challenges and Opportunities](https://mlsanity.com/paper/2010.10777/) (2020)
- [An AI system to help scientists write expert-level empirical software](https://mlsanity.com/paper/2509.06503/) (2025)
- [Number Cookbook: Number Understanding of Language Models and How to Improve It](https://mlsanity.com/paper/2411.03766/) (2024)
- [Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis Diagnosis](https://mlsanity.com/paper/2309.12368/) (2023)
- [Teacher Supervises Students How to Learn From Partially Labeled Images for Facial Landmark Detection](https://mlsanity.com/paper/1908.02116/) (2019)
- [SciSciGPT: Advancing Human-AI Collaboration in the Science of Science](https://mlsanity.com/paper/2504.05559/) (2025)
- [Large Language Models Streamline Automated Machine Learning for Clinical Studies](https://mlsanity.com/paper/2308.14120/) (2023)
- [A Review of Pseudo-Labeling for Computer Vision](https://mlsanity.com/paper/2408.07221/) (2024)
- [Assessing the communication gap between AI models and healthcare professionals: explainability, utility and trust in AI-driven clinical decision-making](https://mlsanity.com/paper/2204.05030/) (2022)
- [Artificial Intelligence Should Genuinely Support Clinical Reasoning and Decision Making To Bridge the Translational Gap](https://mlsanity.com/paper/2506.05030/) (2025)
- [Teach me to segment with mixed supervision: Confident students become masters](https://mlsanity.com/paper/2012.08051/) (2020)
- [A Perspective for Adapting Generalist AI to Specialized Medical AI Applications and Their Challenges](https://mlsanity.com/paper/2411.00024/) (2024)
- [Towards a Medical AI Scientist](https://mlsanity.com/paper/2603.28589/) (2026)
- [Segmentation with mixed supervision: Confidence maximization helps knowledge distillation](https://mlsanity.com/paper/2109.10902/) (2021)
- [Augmenting Chest X-ray Datasets with Non-Expert Annotations](https://mlsanity.com/paper/2309.02244/) (2023)

## Abstract (arXiv)

Developing AI models that are useful in clinical practice, requires efficient collaboration between clinicians and AI developers. This poses a practical challenge: clinicians must repeatedly communicate and refine their requirements with AI developers before those requirements can be translated into executable model development. This iterative process is time-consuming, and even after repeated discussion, misalignment may still exist because the two sides do not fully share each other's expertise. Coding agents may help close this gap. They can write and refine code on their own, and they carry working knowledge of both medicine and AI to understand commands formulated by both medical experts and developers. We present a prototype that lets clinicians drive AI development directly. A clinician describes the task in plain language, and the system turns the description into a working pipeline, refines it through repeated experiments together with the clinician, and returns a model that meets the stated clinical objective. Across five clinical tasks, the system reliably produces models that matched the clinician's request and reached competitive performance. Most notably, on chest radiographs the system sharply reduced the model's reliance on chest drains, a well-known shortcut for pneumothorax classification, from 60% to 31% on one dataset and from 50% to 18% on another. Our results suggest that coding agents can shift clinical AI development toward a more clinician-driven mode, allowing domain experts to shape models directly instead of relaying requirements through specialized AI teams.