mlsanityData to 7 Oct 2026

DrHouse: An LLM-empowered Diagnostic Reasoning System through Harnessing Outcomes from Sensor Data and Expert Knowledge

arXiv 2405.12541 ↗First posted 21 May 2024cs.AIcited by 24

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

Large language models (LLMs) have the potential to transform digital healthcare, as evidenced by recent advances in LLM-based virtual doctors. However, current approaches rely on patient's subjective descriptions of symptoms, causing increased misdiagnosis. Recognizing the value of daily data from smart devices, we introduce a novel LLM-based multi-turn consultation virtual doctor system, DrHouse, which incorporates three significant contributions: 1) It utilizes sensor data from smart devices in the diagnosis process, enhancing accuracy and reliability. 2) DrHouse leverages continuously updating medical databases such as Up-to-Date and PubMed to ensure our model remains at diagnostic standard's forefront. 3) DrHouse introduces a novel diagnostic algorithm that concurrently evaluates potential diseases and their likelihood, facilitating more nuanced and informed medical assessments. Through multi-turn interactions, DrHouse determines the next steps, such as accessing daily data from smart devices or requesting in-lab tests, and progressively refines its diagnoses. Evaluations on three public datasets and our self-collected datasets show that DrHouse can achieve up to an 18.8% increase in diagnosis accuracy over the state-of-the-art baselines. The results of a 32-participant user study show that 75% medical experts and 91.7% patients are willing to use DrHouse.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 30 of its 77 references that are in the map. First posted on arXiv 21 May 2024.

Builds on

30 of 77 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024JMLR: Joint Medical LLM and Retrieval Training for Enhancing Reasoning and Professional Question Answering Capability27 Feb

Other references · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  6. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  7. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  8. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  9. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  10. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  11. 2022Language Models are Realistic Tabular Data Generatorscited by 290
  12. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
17 more
  1. 2024Long-context LLMs Struggle with Long In-context Learningcited by 285
  2. 2023A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Modelscited by 244
  3. 2023HuatuoGPT, towards Taming Language Model to Be a Doctorcited by 211
  4. 2023DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultationcited by 89
  5. 2023LLMs Accelerate Annotation for Medical Information Extractioncited by 77
  6. 2023Penetrative AI: Making LLMs Comprehend the Physical Worldcited by 64
  7. 2024HARGPT: Are LLMs Zero-Shot Human Activity Recognizers?cited by 52
  8. 2020MedDG: An Entity-Centric Medical Consultation Dataset for Entity-Aware Medical Dialogue Generationcited by 45
  9. 2021Semi-Supervised Variational Reasoning for Medical Dialogue Generationcited by 30
  10. 2024VIAssist: Adapting Multi-modal Large Language Models for Users with Visual Impairmentscited by 24
  11. 2024Beyond Direct Diagnosis: LLM-based Multi-Specialist Agent Consultation for Automatic Diagnosiscited by 23
  12. 2023EdgeFM: Leveraging Foundation Model for Open-set Learning on the Edgecited by 21
  13. 2024LLM-based Conversational AI Therapist for Daily Functioning Screening and Psychotherapeutic Intervention via Everyday Smart Devicescited by 21
  14. 2024Evaluating Large Language Models as Virtual Annotators for Time-series Physical Sensing Datacited by 17
  15. 2022DialMed: A Dataset for Dialogue-based Medication Recommendationcited by 12
  16. 2024MESEN: Exploit Multimodal Data to Design Unimodal Human Activity Recognition with Few Labelscited by 11
  17. 2023BrainZ-BP: A Non-invasive Cuff-less Blood Pressure Estimation Approach Leveraging Brain Bio-impedance and Electrocardiogramcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 11; 2026: 7.

24 papers.

Sort:

By area:

  1. 2025MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilotcited by 83
  2. 2024VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledgecited by 50
  3. 2025ContextAgent: Context-Aware Proactive LLM Agents with Open-World Sensory Perceptionscited by 44
  4. 2024SocialMind: LLM-based Proactive AR Social Assistive System with Human-like Perception for In-situ Live Interactionscited by 25
  5. 2025Standardization of Psychiatric Diagnoses -- Role of Fine-tuned LLM Consortium and OpenAI-gpt-oss Reasoning LLM Enabled Decision Support Systemcited by 11
  6. 2025From Screens to Scenes: A Survey of Embodied AI in Healthcarecited by 10
  7. 2025Proof-of-TBI -- Fine-Tuned Vision Language Model Consortium and OpenAI-o3 Reasoning LLM-Based Medical Diagnosis Support System for Mild Traumatic Brain Injury (TBI) Predictioncited by 10
  8. 2025OpenTCM: A GraphRAG-Empowered LLM-based System for Traditional Chinese Medicine Knowledge Retrieval and Diagnosiscited by 10
  9. 2025ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusalcited by 6
  10. 2024Vital Insight: Assisting Experts' Context-Driven Sensemaking of Multi-modal Personal Tracking Data Using Visualization and Human-In-The-Loop LLMcited by 5
  11. 2025Exploring the Capabilities of LLMs for IMU-based Fine-grained Human Activity Understandingcited by 5
  12. 2024Critique of Impure Reason: Unveiling the reasoning behaviour of medical Large Language Modelscited by 2
12 more
  1. 2025SHADE-AD: An LLM-Based Framework for Synthesizing Activity Data of Alzheimer's Patientscited by 2
  2. 2026Toward Zero-Egress Psychiatric AI: On-Device LLM Deployment for Privacy-Preserving Mental Health Decision Supportcited by 2
  3. 2024SensEmo: Enabling Affective Learning through Real-time Emotion Recognition with Smartwatchescited by 1
  4. 2025Design and Evaluation of Generative Agent-based Platform for Human-Assistant Interaction Research: A Tale of 10 User Studiescited by 1
  5. 2024See Where You Read with Eye Gaze Tracking and Large Language Modelcited by 0
  6. 2025DeepFeature: LLM-Empowered Context-aware Feature Generation for Wearable Biosignalscited by 0
  7. 2026After Talking with 1,000 Personas: Learning Preference-Aligned Proactive Assistants From Large-Scale Persona Interactionscited by 0
  8. 2026Evaluating Visual Prompts with Eye-Tracking Data for MLLM-Based Human Activity Recognitioncited by 0
  9. 2026Train the Trainers -- An Agentic AI Framework for Peer-Based Mental Health Support in Battlefield Environmentscited by 0
  10. 2026From 'What' to 'How' and 'Why': Sharing LLM-Generated Retrospective Summaries of Older Adults' Passive Tracking Data with Remote Family Memberscited by 0
  11. 2026MedGuards: Multi-Agent System for Reliable Medical Error Detection and Correctioncited by 0
  12. 2026Trustworthy AI in Digital Health: A Comprehensive Review of Robustness and Explainabilitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Aligning Clinical Needs and AI Capabilities: A Survey on LLMs for Medical Reasoning
  2. 2025Medicine on the Edge: Comparative Performance Analysis of On-Device LLMs for Clinical Reasoning
  3. 2023Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review
  4. 2025Medical Reasoning in LLMs: An In-Depth Analysis of DeepSeek R1
  5. 2023Towards Accurate Differential Diagnosis with Large Language Models
  6. 2025Reasoning LLMs in the Medical Domain: A Literature Survey
  7. 2023Leveraging Medical Knowledge Graphs Into Large Language Models for Diagnosis Prediction: Design and Application Study
  8. 2024Superhuman performance of a large language model on the reasoning tasks of a physician