The Impact of VR and 2D Interfaces on Human Feedback in Preference-Based Robot Learning
On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF
Abstract (arXiv)
Aligning robot navigation with human preferences is essential for ensuring comfortable, and predictable robot movement in shared spaces. While preference-based learning methods, such as reinforcement learning from human feedback (RLHF), enable this alignment, the choice of the preference collection interface may influence the process. Traditional 2D interfaces provide structured views but lack spatial depth, whereas immersive VR offers richer perception, potentially affecting preference articulation. This study systematically examines how the interface modality impacts human preference collection and navigation policy alignment. We introduce a novel dataset of 2,325 human preference queries collected through both VR and 2D interfaces, revealing significant differences in user experience, preference consistency, and policy outcomes. Our findings highlight the trade-offs between immersion, perception, and preference reliability, emphasizing the importance of interface selection in preference-based robot learning. The dataset is available to support future research.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 15 of its 35 references that are in the map. First posted on arXiv 11 Mar 2025.
Builds on
15 of 35 references.
Closest ancestors · 5
Same direction, newest first
- 2024PREDILECT: Preferences Delineated with Zero-Shot Language-based Reasoning in Reinforcement Learning23 Feb
- 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback27 Jul
- 2019Fine-Tuning Language Models from Human Preferences18 Sep
- 2018Reward learning from human preferences and demonstrations in Atari15 Nov
- 2017Deep reinforcement learning from human preferences12 Jun
Other references · 10
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
- 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 194
- 2022Few-Shot Preference Learning for Human-in-the-Loop RLcited by 87
- 2019Learning Reward Functions by Integrating Human Demonstrations and Preferencescited by 64
- 2017Repeated Inverse Reinforcement Learningcited by 29
- 2022Feedback-efficient Active Preference Learning for Socially Aware Robot Navigationcited by 7
- 2022Learning Depth Vision-Based Personalized Robot Navigation From Dynamic Demonstrations in Virtual Realitycited by 5
- 2020Towards an immersive user interface for waypoint navigation of a mobile robotcited by 2
- 2023Happily Error After: Framework Development and User Study for Correcting Robot Perception Errors in Virtual Realitycited by 1
- 2024EnQuery: Ensemble Policies for Diverse Query-Generation in Preference Alignment of Robot Navigationcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Survey
- 2025PRISM: Preference Refinement via Implicit Scene Modeling for 3D Vision-Language Preference-Based Reinforcement Learning
- 2026Interpreting Context-Aware Human Preferences for Multi-Objective Robot Navigation ↗
- 2024Maximizing Alignment with Minimal Feedback: Efficiently Learning Rewards for Visuomotor Robot Policy Alignment
- 2024PrefMMT: Modeling Human Preferences in Preference-based Reinforcement Learning with Multimodal Transformers
- 2026Preference Aligned Visuomotor Diffusion Policies for Deformable Object Manipulation
- 2026SPHERE: Adaptive VR Indoor Scene Generation via LLM-Enhanced Spatial Preference Learning and Human-in-the-Loop RL
- 2024Demonstration-Enhanced Adaptable Multi-Objective Robot Navigation