mlsanityData to 14 Oct 2026

Immersive Explainability: Visualizing Robot Navigation Decisions through XAI Semantic Scene Projections in Virtual Reality

arXiv 2504.00682 ↗First posted 1 Apr 2025cs.ROcited by 0

On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL

Abstract (arXiv)

End-to-end robot policies achieve high performance through neural networks trained via reinforcement learning (RL). Yet, their black box nature and abstract reasoning pose challenges for human-robot interaction (HRI), because humans may experience difficulty in understanding and predicting the robot's navigation decisions, hindering trust development. We present a virtual reality (VR) interface that visualizes explainable AI (XAI) outputs and the robot's lidar perception to support intuitive interpretation of RL-based navigation behavior. By visually highlighting objects based on their attribution scores, the interface grounds abstract policy explanations in the scene context. This XAI visualization bridges the gap between obscure numerical XAI attribution scores and a human-centric semantic level of explanation. A within-subjects study with 24 participants evaluated the effectiveness of our interface for four visualization conditions combining XAI and lidar. Participants ranked scene objects across navigation scenarios based on their importance to the robot, followed by a questionnaire assessing subjective understanding and predictability. Results show that semantic projection of attributions significantly enhances non-expert users' objective understanding and subjective awareness of robot behavior. In addition, lidar visualization further improves perceived predictability, underscoring the value of integrating XAI and sensor for transparent, trustworthy HRI.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 16 of its 28 references that are in the map. First posted on arXiv 1 Apr 2025.

Builds on

16 of 28 references.

References · 16

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  6. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
  7. 2018Metrics for Explainable AI: Challenges and Prospectscited by 260
  8. 2020Questioning the AI: Informing Design Practices for Explainable AI User Experiencescited by 253
  9. 2020A Survey on the Explainability of Supervised Machine Learningcited by 227
  10. 2020Explainable Reinforcement Learning: A Surveycited by 141
  11. 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 85
  12. 2018Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systemscited by 48
4 more
  1. 2021Semantic-Based Explainable AI: Leveraging Semantic Scene Graphs and Pairwise Ranking to Explain Robot Failurescited by 13
  2. 2022BASED-XAI: Breaking Ablation Studies Down for Explainable Artificial Intelligencecited by 13
  3. 2023Explainable Human-Robot Training and Cooperation with Augmented Realitycited by 7
  4. 2024Robot Explanation Identitycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Abstracted Trajectory Visualization for Explainability in Reinforcement Learning
  2. 2026Explainable OOHRI: Communicating Robot Capabilities and Limitations as Augmented Reality Affordances ↗
  3. 2026Explainable AI for Blind and Low-Vision Users: Navigating Trust, Modality, and Interpretability in the Agentic Era
  4. 2025Trust Through Transparency: Explainable Social Navigation for Autonomous Mobile Robots via Vision-Language Models
  5. 2022Decisions that Explain Themselves: A User-Centric Deep Reinforcement Learning Explanation System
  6. 2026Interactive Navigation Beyond Reachability: Benchmark and Agentic Framework
  7. 2024Enhancing Robot Explanation Capabilities through Vision-Language Models: a Preliminary Study by Interpreting Visual Inputs for Improved Human-Robot Interaction
  8. 2026What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior