mlsanityData to 14 Oct 2026

Adaptive 3D UI Placement in Mixed Reality Using Deep Reinforcement Learning

arXiv 2504.21731 ↗First posted 30 Apr 2025cs.HC cs.AI cs.CVcited by 1

On the map: Deep reinforcement learning methods → RL with diffusion and flow models

Abstract (arXiv)

Mixed Reality (MR) could assist users' tasks by continuously integrating virtual content with their view of the physical environment. However, where and how to place these content to best support the users has been a challenging problem due to the dynamic nature of MR experiences. In contrast to prior work that investigates optimization-based methods, we are exploring how reinforcement learning (RL) could assist with continuous 3D content placement that is aware of users' poses and their surrounding environments. Through an initial exploration and preliminary evaluation, our results demonstrate the potential of RL to position content that maximizes the reward for users on the go. We further identify future directions for research that could harness the power of RL for personalized and optimized UI and content placement in MR.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods. Builds on 15 of its 47 references that are in the map. First posted on arXiv 30 Apr 2025.

Builds on

15 of 47 references.

References · 15

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Deep reinforcement learning from human preferencescited by 3,912
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  5. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  6. 2019Model-Based Reinforcement Learning for Ataricited by 502
  7. 2018Unity: A General Platform for Intelligent Agentscited by 236
  8. 2018Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learningcited by 175
  9. 2022Few-Shot Preference Learning for Human-in-the-Loop RLcited by 87
  10. 2020Chip Placement with Deep Reinforcement Learningcited by 83
  11. 2021Adapting User Interfaces with Model-based Reinforcement Learningcited by 17
  12. 2021On the Use and Misuse of Absorbing States in Multi-agent Reinforcement Learningcited by 10
3 more
  1. 2020GRIDS: Interactive Layout Design with Integer Programmingcited by 9
  2. 2019External Labeling Techniques: A Taxonomy and Surveycited by 2
  3. 2021Context-Responsive Labeling in Augmented Realitycited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Inverse Theory of Mind Modeling for Content Recommendation: From Web Browsing to Dynamic Intelligent Interfacescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advances
  2. 2025MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs for the Metaverse
  3. 2026SPHERE: Adaptive VR Indoor Scene Generation via LLM-Enhanced Spatial Preference Learning and Human-in-the-Loop RL
  4. 2025Are We Ready for RL in Text-to-3D Generation? A Progressive Investigation
  5. 2026Scaling Sim-to-Real VLA Reinforcement Learning with Generative 3D Worlds
  6. 2025Boosting Reinforcement Learning in 3D Visuospatial Tasks Through Human-Informed Curriculum Design
  7. 2024Physics-based Scene Layout Generation from Human Motion
  8. 2024R2-Mesh: Reinforcement Learning Powered Mesh Reconstruction via Geometry and Appearance Refinement