# VirtuWander: Enhancing Multi-modal Interaction for Virtual Tour Guidance through Large Language Models

arXiv: https://arxiv.org/abs/2401.11923 · HTML: https://mlsanity.com/paper/2401.11923/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jan 2024.

## Built on by (most cited first, 7 of 7)

- [Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attacks](https://mlsanity.com/paper/2409.08087/) (2024)
- [Classroom Simulacra: Building Contextual Student Generative Agents in Online Education for Learning Behavioral Simulation](https://mlsanity.com/paper/2502.02780/) (2025)
- [Mitigating Response Delays in Free-Form Conversations with LLM-powered Intelligent Virtual Agents](https://mlsanity.com/paper/2507.22352/) (2025)
- [Multimodal Cultural Safety: Evaluation Framework and Alignment Strategies](https://mlsanity.com/paper/2505.14972/) (2025)
- [CTourLLM: Enhancing LLMs with Chinese Tourism Knowledge](https://mlsanity.com/paper/2407.12791/) (2024)
- [Navigation Pixie: Implementation and Empirical Study Toward On-demand Navigation Agents in Commercial Metaverse](https://mlsanity.com/paper/2508.03216/) (2025)
- [SimViews: An Interactive Multi-Agent System Simulating Visitor-to-Visitor Conversational Patterns to Present Diverse Perspectives of Artifacts in Virtual Museums](https://mlsanity.com/paper/2508.07730/) (2025)

## Abstract (arXiv)

Tour guidance in virtual museums encourages multi-modal interactions to boost user experiences, concerning engagement, immersion, and spatial awareness. Nevertheless, achieving the goal is challenging due to the complexity of comprehending diverse user needs and accommodating personalized user preferences. Informed by a formative study that characterizes guidance-seeking contexts, we establish a multi-modal interaction design framework for virtual tour guidance. We then design VirtuWander, a two-stage innovative system using domain-oriented large language models to transform user inquiries into diverse guidance-seeking contexts and facilitate multi-modal interactions. The feasibility and versatility of VirtuWander are demonstrated with virtual guiding examples that encompass various touring scenarios and cater to personalized preferences. We further evaluate VirtuWander through a user study within an immersive simulated museum. The results suggest that our system enhances engaging virtual tour experiences through personalized communication and knowledgeable assistance, indicating its potential for expanding into real-world scenarios.