# Agentic AI framework for End-to-End Medical Data Inference

arXiv: https://arxiv.org/abs/2507.18115 · HTML: https://mlsanity.com/paper/2507.18115/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 23 of its 55 references that are in the map.
- First posted on arXiv 24 Jul 2025.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 23 of 23)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [The Future of Digital Health with Federated Learning](https://mlsanity.com/paper/2003.08119/) (2020)
- [AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors](https://mlsanity.com/paper/2308.10848/) (2023)
- [MedGemma Technical Report](https://mlsanity.com/paper/2507.05201/) (2025)
- [AutoML: A Survey of the State-of-the-Art](https://mlsanity.com/paper/1908.00709/) (2019)
- [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020)
- [Self-Alignment Pretraining for Biomedical Entity Representations](https://mlsanity.com/paper/2010.11784/) (2020)
- [Can large language models reason about medical questions?](https://mlsanity.com/paper/2207.08143/) (2022)
- [Auto-Keras: An Efficient Neural Architecture Search System](https://mlsanity.com/paper/1806.10282/) (2018)
- [The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey](https://mlsanity.com/paper/2404.11584/) (2024)
- [Differential Privacy-enabled Federated Learning for Sensitive Health Data](https://mlsanity.com/paper/1910.02578/) (2019)
- [Colonoscopy Polyp Detection and Classification: Dataset Creation and Comparative Evaluations](https://mlsanity.com/paper/2104.10824/) (2021)
- [OBOE: Collaborative Filtering for AutoML Model Selection](https://mlsanity.com/paper/1808.03233/) (2018)
- [ELT-Bench: An End-to-End Benchmark for Evaluating AI Agents on ELT Pipelines](https://mlsanity.com/paper/2504.04808/) (2025)
- [Towards a HIPAA Compliant Agentic AI System in Healthcare](https://mlsanity.com/paper/2504.17669/) (2025)
- [Magika: AI-Powered Content-Type Detection](https://mlsanity.com/paper/2409.13768/) (2024)
- [Intelligent Spark Agents: A Modular LangGraph Framework for Scalable, Visualized, and Enhanced Big Data Machine Learning Workflows](https://mlsanity.com/paper/2412.01490/) (2024)
- [Causal State Distillation for Explainable Reinforcement Learning](https://mlsanity.com/paper/2401.00104/) (2023)
- [LLM-Match: An Open-Sourced Patient Matching Model Based on Large Language Models and Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.13281/) (2025)
- [Bel Esprit: Multi-Agent Framework for Building AI Model Pipelines](https://mlsanity.com/paper/2412.14684/) (2024)

## Built on by (most cited first, 8 of 8)

- [AT-CXR: Uncertainty-Aware Agentic Triage for Chest X-rays](https://mlsanity.com/paper/2508.19322/) (2025)
- [One VLM, Two Roles: Stage-Wise Routing and Specialty-Level Deployment for Clinical Workflows](https://mlsanity.com/paper/2508.16839/) (2025)
- [OpenLens AI: Fully Autonomous Research Agent for Health Infomatics](https://mlsanity.com/paper/2509.14778/) (2025)
- [The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomy](https://mlsanity.com/paper/2607.11175/) (2026)
- [From Wearables to Warnings: Predicting Pain Spikes in Patients with Opioid Use Disorder](https://mlsanity.com/paper/2511.19577/) (2025)
- [Belief Is All You Need: Modeling Narrative Archetypes in Conspiratorial Discourse](https://mlsanity.com/paper/2512.10105/) (2025)
- [Why Specialist Models Still Matter: A Heterogeneous Multi-Agent Paradigm for Medical Artificial Intelligence](https://mlsanity.com/paper/2605.29744/) (2026)
- [Tracing the Heart: An Evidence-Linked Pipeline for Heart-Failure Feature Engineering](https://mlsanity.com/paper/2608.06366/) (2026)

## Abstract (arXiv)

Building and deploying machine learning solutions in healthcare remains expensive and labor-intensive due to fragmented preprocessing workflows, model compatibility issues, and stringent data privacy constraints. In this work, we introduce an Agentic AI framework that automates the entire clinical data pipeline, from ingestion to inference, through a system of modular, task-specific agents. These agents handle both structured and unstructured data, enabling automatic feature selection, model selection, and preprocessing recommendation without manual intervention. We evaluate the system on publicly available datasets from geriatrics, palliative care, and colonoscopy imaging. For example, in the case of structured data (anxiety data) and unstructured data (colonoscopy polyps data), the pipeline begins with file-type detection by the Ingestion Identifier Agent, followed by the Data Anonymizer Agent ensuring privacy compliance, where we first identify the data type and then anonymize it. The Feature Extraction Agent identifies features using an embedding-based approach for tabular data, extracting all column names, and a multi-stage MedGemma-based approach for image data, which infers modality and disease name. These features guide the Model-Data Feature Matcher Agent in selecting the best-fit model from a curated repository. The Preprocessing Recommender Agent and Preprocessing Implementor Agent then apply tailored preprocessing based on data type and model requirements. Finally, the ``Model Inference Agent" runs the selected model on the uploaded data and generates interpretable outputs using tools like SHAP, LIME, and DETR attention maps. By automating these high-friction stages of the ML lifecycle, the proposed framework reduces the need for repeated expert intervention, offering a scalable, cost-efficient pathway for operationalizing AI in clinical environments.