# Exploring User Acceptance and Concerns toward LLM-powered Conversational Agents in Immersive Extended Reality

arXiv: https://arxiv.org/abs/2512.15343 · HTML: https://mlsanity.com/paper/2512.15343/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 56 references that are in the map.
- First posted on arXiv 17 Dec 2025.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 17 of 17)

- [Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenes](https://mlsanity.com/paper/2308.08769/) (2023)
- ["It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents](https://mlsanity.com/paper/2309.11653/) (2023)
- [LLMR: Real-time Prompting of Interactive Worlds using Large Language Models](https://mlsanity.com/paper/2309.12276/) (2023)
- [CloChat: Understanding How People Customize, Interact, and Experience Personas in Large Language Models](https://mlsanity.com/paper/2402.15265/) (2024)
- [Embedding Large Language Models into Extended Reality: Opportunities and Challenges for Inclusion, Engagement, and Privacy](https://mlsanity.com/paper/2402.03907/) (2024)
- [Privacy-Aware Eye Tracking Using Differential Privacy](https://mlsanity.com/paper/1812.08000/) (2018)
- [Digital Transformations of Classrooms in Virtual Reality](https://mlsanity.com/paper/2101.09576/) (2021)
- [Developing a Personality Model for Speech-based Conversational Agents Using the Psycholexical Approach](https://mlsanity.com/paper/2003.06186/) (2020)
- [Like My Aunt Dorothy: Effects of Conversational Styles on Perceptions, Acceptance and Metaphorical Descriptions of Voice Assistants during Later Adulthood](https://mlsanity.com/paper/2309.11681/) (2023)
- [LLMER: Crafting Interactive Extended Reality Worlds with JSON Data Generated by Large Language Models](https://mlsanity.com/paper/2502.02441/) (2025)
- [Real-time Animation Generation and Control on Rigged Models via Large Language Models](https://mlsanity.com/paper/2310.17838/) (2023)
- [CUIfy the XR: An Open-Source Package to Embed LLM-powered Conversational Agents in XR](https://mlsanity.com/paper/2411.04671/) (2024)
- [Evaluating Usability and Engagement of Large Language Models in Virtual Reality for Traditional Scottish Curling](https://mlsanity.com/paper/2408.09285/) (2024)
- [Assessing User Apprehensions About Mixed Reality Artifacts and Applications: The Mixed Reality Concerns (MRC) Questionnaire](https://mlsanity.com/paper/2403.05855/) (2024)
- [DataliVR: Transformation of Data Literacy Education through Virtual Reality with ChatGPT-Powered Enhancements](https://mlsanity.com/paper/2409.08738/) (2024)
- [LLM-Mediated Domain-Specific Voice Agents: The Case of TextileBot](https://mlsanity.com/paper/2406.10590/) (2024)
- [The Impact of Device Type, Data Practices, and Use Case Scenarios on Privacy Concerns about Eye-tracked Augmented Reality in the United States and Germany](https://mlsanity.com/paper/2509.09285/) (2025)

## Abstract (arXiv)

The rapid development of generative artificial intelligence (AI) and large language models (LLMs), and the availability of services that make them accessible, have led the general public to begin incorporating them into everyday life. The extended reality (XR) community has also sought to integrate LLMs, particularly in the form of conversational agents, to enhance user experience and task efficiency. When interacting with such conversational agents, users may easily disclose sensitive information due to the naturalistic flow of the conversations, and combining such conversational data with fine-grained sensor data may lead to novel privacy issues. To address these issues, a user-centric understanding of technology acceptance and concerns is essential. Therefore, to this end, we conducted a large-scale crowdsourcing study with 1036 participants, examining user decision-making processes regarding LLM-powered conversational agents in XR, across factors of XR setting type, speech interaction type, and data processing location. We found that while users generally accept these technologies, they express concerns related to security, privacy, social implications, and trust. Our results suggest that familiarity plays a crucial role, as daily generative AI use is associated with greater acceptance. In contrast, previous ownership of XR devices is linked to less acceptance, possibly due to existing familiarity with the settings. We also found that men report higher acceptance with fewer concerns than women. Regarding data type sensitivity, location data elicited the most significant concern, while body temperature and virtual object states were considered least sensitive. Overall, our study highlights the importance of practitioners effectively communicating their measures to users, who may remain distrustful. We conclude with implications and recommendations for LLM-powered XR.