# Mitigating Response Delays in Free-Form Conversations with LLM-powered Intelligent Virtual Agents

arXiv: https://arxiv.org/abs/2507.22352 · HTML: https://mlsanity.com/paper/2507.22352/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 6 of its 114 references that are in the map.
- First posted on arXiv 30 Jul 2025.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 6 of 6)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Virtual Agents for Alcohol Use Counseling: Exploring LLM-Powered Motivational Interviewing](https://mlsanity.com/paper/2407.08095/) (2024)
- [An Aligned Rank Transform Procedure for Multifactor Contrast Tests](https://mlsanity.com/paper/2102.11824/) (2021)
- [VirtuWander: Enhancing Multi-modal Interaction for Virtual Tour Guidance through Large Language Models](https://mlsanity.com/paper/2401.11923/) (2024)
- [Takeaways from Applying LLM Capabilities to Multiple Conversational Avatars in a VR Pilot Study](https://mlsanity.com/paper/2501.00168/) (2024)
- [VALID: A perceptually validated Virtual Avatar Library for Inclusion and Diversity](https://mlsanity.com/paper/2309.10902/) (2023)

## Built on by (most cited first, 12 of 12)

- [SnapGuard: Lightweight Prompt Injection Detection for Screenshot-Based Web Agents](https://mlsanity.com/paper/2604.25562/) (2026)
- [Hear You in Silence: Designing for Active Listening in Human Interaction with Conversational Agents Using Context-Aware Pacing](https://mlsanity.com/paper/2602.06134/) (2026)
- [Chaplains' Reflections on the Design and Usage of AI for Conversational Care](https://mlsanity.com/paper/2602.04017/) (2026)
- [What Are You Doing? Effects of Intermediate Feedback from Agentic LLM In-Car Assistants During Multi-Step Processing](https://mlsanity.com/paper/2602.15569/) (2026)
- [RPU -- A Reasoning Processing Unit](https://mlsanity.com/paper/2602.18568/) (2026)
- [DuCCAE: A Hybrid Engine for Immersive Conversation via Collaboration, Augmentation, and Evolution](https://mlsanity.com/paper/2603.19248/) (2026)
- [TurnNat: Automatic Evaluation of Turn-Taking Naturalness in Dyadic Spoken Dialogue](https://mlsanity.com/paper/2607.01345/) (2026)
- [Dynamic Multimodal Expression Generation for LLM-Driven Pedagogical Agents: From User Experience Perspective](https://mlsanity.com/paper/2603.09536/) (2026)
- [RAGnaroX: A Secure, Local-Hosted ChatOps Assistant Using Small Language Models](https://mlsanity.com/paper/2604.03291/) (2026)
- [From Prompt to Service: An SLM-Based Agent Orchestration Gateway for AI-Driven Virtual Worlds](https://mlsanity.com/paper/2606.03557/) (2026)
- [Low-Latency Turn-Taking via Context-Aware Preface Generation in a Real-World Dialogue Robot](https://mlsanity.com/paper/2607.23204/) (2026)
- [Gricea: An Open Science Platform for Conversational AI Research](https://mlsanity.com/paper/2609.22039/) (2026)

## Abstract (arXiv)

We investigated the challenges of mitigating response delays in free-form conversations with virtual agents powered by Large Language Models (LLMs) within Virtual Reality (VR). For this, we used conversational fillers, such as gestures and verbal cues, to bridge delays between user input and system responses and evaluate their effectiveness across various latency levels and interaction scenarios. We found that latency above 4 seconds degrades quality of experience, while natural conversational fillers improve perceived response time, especially in high-delay conditions. Our findings provide insights for practitioners and researchers to optimize user engagement whenever conversational systems' responses are delayed by network limitations or slow hardware. We also contribute an open-source pipeline that streamlines deploying conversational agents in virtual environments.