# A Smart Multimodal Healthcare Copilot with Powerful LLM Reasoning

arXiv: https://arxiv.org/abs/2506.02470 · HTML: https://mlsanity.com/paper/2506.02470/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 27 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Builds on (most cited first, 10 of 10)

- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge](https://mlsanity.com/paper/2303.14070/) (2023)
- [Medical Graph RAG: Towards Safe Medical Large Language Model via Graph Retrieval-Augmented Generation](https://mlsanity.com/paper/2408.04187/) (2024)
- [MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilot](https://mlsanity.com/paper/2502.04413/) (2025)
- [Healthcare Copilot: Eliciting the Power of General LLMs for Medical Consultation](https://mlsanity.com/paper/2402.13408/) (2024)
- [Brain-inspired Search Engine Assistant based on Knowledge Graph](https://mlsanity.com/paper/2012.13529/) (2020)
- [A Generalizable Anomaly Detection Method in Dynamic Graphs](https://mlsanity.com/paper/2412.16447/) (2024)
- [Almanac Copilot: Towards Autonomous Electronic Health Record Navigation](https://mlsanity.com/paper/2405.07896/) (2024)
- [A Survey of Artificial Intelligence in Gait-Based Neurodegenerative Disease Diagnosis](https://mlsanity.com/paper/2405.13082/) (2024)

## Built on by (most cited first, 3 of 3)

- [GFRIEND: Generative Few-shot Reward Inference through EfficieNt DPO](https://mlsanity.com/paper/2506.08965/) (2025)
- [Are Heterogeneous Graph Neural Networks Truly Effective for Node Classification? A Causal Perspective](https://mlsanity.com/paper/2510.05750/) (2025)
- [GEM: Generative Entropy-Guided Preference Modeling for Few-shot Alignment of LLMs](https://mlsanity.com/paper/2511.13007/) (2025)

## Abstract (arXiv)

Misdiagnosis causes significant harm to healthcare systems worldwide, leading to increased costs and patient risks. MedRAG is a smart multimodal healthcare copilot equipped with powerful large language model (LLM) reasoning, designed to enhance medical decision-making. It supports multiple input modalities, including non-intrusive voice monitoring, general medical queries, and electronic health records. MedRAG provides recommendations on diagnosis, treatment, medication, and follow-up questioning. Leveraging retrieval-augmented generation enhanced by knowledge graph-elicited reasoning, MedRAG retrieves and integrates critical diagnostic insights, reducing the risk of misdiagnosis. It has been evaluated on both public and private datasets, outperforming existing models and offering more specific and accurate healthcare assistance. A demonstration video of MedRAG is available at: https://www.youtube.com/watch?v=PNIBDMYRfDM. The source code is available at: https://github.com/SNOWTEAM2023/MedRAG.