# Designing VR Simulation System for Clinical Communication Training with LLMs-Based Embodied Conversational Agents

arXiv: https://arxiv.org/abs/2503.01767 · HTML: https://mlsanity.com/paper/2503.01767/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2025.

## Built on by (most cited first, 2 of 2)

- [How LLMs are Shaping the Future of Virtual Reality](https://mlsanity.com/paper/2508.00737/) (2025)
- [Reading the Mood Behind Words: Integrating Prosody-Derived Emotional Context into Socially Responsive VR Agents](https://mlsanity.com/paper/2603.09324/) (2026)

## Abstract (arXiv)

VR simulation in Health Professions (HP) education demonstrates huge potential, but fixed learning content with little customization limits its application beyond lab environments. To address these limitations in the context of VR for patient communication training, we conducted a user-centered study involving semi-structured interviews with advanced HP students to understand their challenges in clinical communication training and perceptions of VR-based solutions. From this, we derived design insights emphasizing the importance of realistic scenarios, simple interactions, and unpredictable dialogues. Building on these insights, we developed the Virtual AI Patient Simulator (VAPS), a novel VR system powered by Large Language Models (LLMs) and Embodied Conversational Agents (ECAs), supporting dynamic and customizable patient interactions for immersive learning. We also provided an example of how clinical professors could use user-friendly design forms to create personalized scenarios that align with course objectives in VAPS and discuss future implications of integrating AI-driven technologies into VR education.