# Healthcare Copilot: Eliciting the Power of General LLMs for Medical Consultation

arXiv: https://arxiv.org/abs/2402.13408 · HTML: https://mlsanity.com/paper/2402.13408/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 20 Feb 2024.

## Builds on (most cited first, 28 of 28)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback](https://mlsanity.com/paper/2305.14387/) (2023)
- [ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge](https://mlsanity.com/paper/2303.14070/) (2023)
- [MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Data](https://mlsanity.com/paper/2304.08247/) (2023)
- [HuatuoGPT, towards Taming Language Model to Be a Doctor](https://mlsanity.com/paper/2305.15075/) (2023)
- [A Study of Generative Large Language Model for Medical Research and Healthcare](https://mlsanity.com/paper/2305.13523/) (2023)
- [A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics](https://mlsanity.com/paper/2310.05694/) (2023)
- [Towards Making the Most of ChatGPT for Machine Translation](https://mlsanity.com/paper/2303.13780/) (2023)
- [DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Task](https://mlsanity.com/paper/2304.01097/) (2023)
- [Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialogue](https://mlsanity.com/paper/2308.03549/) (2023)
- [Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding](https://mlsanity.com/paper/2401.12954/) (2024)
- [Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models](https://mlsanity.com/paper/2303.13809/) (2023)
- [SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters](https://mlsanity.com/paper/2210.04284/) (2022)
- [BianQue: Balancing the Questioning and Suggestion Ability of Health LLMs with Multi-turn Health Conversations Polished by ChatGPT](https://mlsanity.com/paper/2310.15896/) (2023)
- [Prompted LLMs as Chatbot Modules for Long Open-domain Conversation](https://mlsanity.com/paper/2305.04533/) (2023)
- [Revisiting Demonstration Selection Strategies in In-Context Learning](https://mlsanity.com/paper/2401.12087/) (2024)
- [Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Tasks](https://mlsanity.com/paper/2310.13800/) (2023)
- [On Efficient Training of Large-Scale Deep Learning Models: A Literature Review](https://mlsanity.com/paper/2304.03589/) (2023)
- [HELP ME THINK: A Simple Prompting Strategy for Non-experts to Create Customized Content with Models](https://mlsanity.com/paper/2208.08232/) (2022)

## Built on by (most cited first, 17 of 17)

- [Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey](https://mlsanity.com/paper/2503.12605/) (2025)
- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilot](https://mlsanity.com/paper/2502.04413/) (2025)
- [Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction](https://mlsanity.com/paper/2403.09963/) (2024)
- [MIRAGE: A Benchmark for Multimodal Information-Seeking and Reasoning in Agricultural Expert-Guided Conversations](https://mlsanity.com/paper/2506.20100/) (2025)
- [Multi-OphthaLingua: A Multilingual Benchmark for Assessing and Debiasing LLM Ophthalmological QA in LMICs](https://mlsanity.com/paper/2412.14304/) (2024)
- [ICLShield: Exploring and Mitigating In-Context Learning Backdoor Attacks](https://mlsanity.com/paper/2507.01321/) (2025)
- [Enabling Scalable Evaluation of Bias Patterns in Medical LLMs](https://mlsanity.com/paper/2410.14763/) (2024)
- [Building Accurate Translation-Tailored LLMs with Language Aware Instruction Tuning](https://mlsanity.com/paper/2403.14399/) (2024)
- [A Smart Multimodal Healthcare Copilot with Powerful LLM Reasoning](https://mlsanity.com/paper/2506.02470/) (2025)
- [360^circREA: Towards A Reusable Experience Accumulation with 360deg Assessment for Multi-Agent System](https://mlsanity.com/paper/2404.05569/) (2024)
- [EEG-MedRAG: Enhancing EEG-based Clinical Decision-Making via Hierarchical Hypergraph Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.13735/) (2025)
- [GuidelineGuard: An Agentic Framework for Medical Note Evaluation with Guideline Adherence](https://mlsanity.com/paper/2411.06264/) (2024)
- [PeerCoPilot: A Language Model-Powered Assistant for Behavioral Health Organizations](https://mlsanity.com/paper/2511.21721/) (2025)
- [What Makes a Good Doctor Response? A Study on Text-Based Telemedicine](https://mlsanity.com/paper/2602.17194/) (2026)
- [It Matters How You Say It: Exploring Rhetorical Patterns for AI-Assisted Information Evaluation](https://mlsanity.com/paper/2607.17627/) (2026)
- [Large language models in medical time series analysis](https://mlsanity.com/paper/2609.22262/) (2026)

## Abstract (arXiv)

The copilot framework, which aims to enhance and tailor large language models (LLMs) for specific complex tasks without requiring fine-tuning, is gaining increasing attention from the community. In this paper, we introduce the construction of a Healthcare Copilot designed for medical consultation. The proposed Healthcare Copilot comprises three main components: 1) the Dialogue component, responsible for effective and safe patient interactions; 2) the Memory component, storing both current conversation data and historical patient information; and 3) the Processing component, summarizing the entire dialogue and generating reports. To evaluate the proposed Healthcare Copilot, we implement an auto-evaluation scheme using ChatGPT for two roles: as a virtual patient engaging in dialogue with the copilot, and as an evaluator to assess the quality of the dialogue. Extensive results demonstrate that the proposed Healthcare Copilot significantly enhances the capabilities of general LLMs for medical consultations in terms of inquiry capability, conversational fluency, response accuracy, and safety. Furthermore, we conduct ablation studies to highlight the contribution of each individual module in the Healthcare Copilot. Code will be made publicly available on GitHub.