# The Impact of VR and 2D Interfaces on Human Feedback in Preference-Based Robot Learning

arXiv: https://arxiv.org/abs/2503.16500 · HTML: https://mlsanity.com/paper/2503.16500/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 15 of its 35 references that are in the map.
- First posted on arXiv 11 Mar 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Reward modeling and RLHF](https://mlsanity.com/alignment/reward-modeling-and-rlhf/)

## Builds on (most cited first, 15 of 15)

- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [Reward learning from human preferences and demonstrations in Atari](https://mlsanity.com/paper/1811.06521/) (2018)
- [iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks](https://mlsanity.com/paper/2108.03272/) (2021)
- [Few-Shot Preference Learning for Human-in-the-Loop RL](https://mlsanity.com/paper/2212.03363/) (2022)
- [Learning Reward Functions by Integrating Human Demonstrations and Preferences](https://mlsanity.com/paper/1906.08928/) (2019)
- [Repeated Inverse Reinforcement Learning](https://mlsanity.com/paper/1705.05427/) (2017)
- [PREDILECT: Preferences Delineated with Zero-Shot Language-based Reasoning in Reinforcement Learning](https://mlsanity.com/paper/2402.15420/) (2024)
- [Feedback-efficient Active Preference Learning for Socially Aware Robot Navigation](https://mlsanity.com/paper/2201.00469/) (2022)
- [Learning Depth Vision-Based Personalized Robot Navigation From Dynamic Demonstrations in Virtual Reality](https://mlsanity.com/paper/2210.01683/) (2022)
- [Towards an immersive user interface for waypoint navigation of a mobile robot](https://mlsanity.com/paper/2003.12772/) (2020)
- [Happily Error After: Framework Development and User Study for Correcting Robot Perception Errors in Virtual Reality](https://mlsanity.com/paper/2306.14589/) (2023)
- [EnQuery: Ensemble Policies for Diverse Query-Generation in Preference Alignment of Robot Navigation](https://mlsanity.com/paper/2404.04852/) (2024)

## Built on by (most cited first, 1 of 1)

- [Generating Natural and Expressive Robot Gestures through Iterative Reinforcement Learning with Human Feedback using LLMs](https://mlsanity.com/paper/2606.18747/) (2026)

## Abstract (arXiv)

Aligning robot navigation with human preferences is essential for ensuring comfortable, and predictable robot movement in shared spaces. While preference-based learning methods, such as reinforcement learning from human feedback (RLHF), enable this alignment, the choice of the preference collection interface may influence the process. Traditional 2D interfaces provide structured views but lack spatial depth, whereas immersive VR offers richer perception, potentially affecting preference articulation. This study systematically examines how the interface modality impacts human preference collection and navigation policy alignment. We introduce a novel dataset of 2,325 human preference queries collected through both VR and 2D interfaces, revealing significant differences in user experience, preference consistency, and policy outcomes. Our findings highlight the trade-offs between immersion, perception, and preference reliability, emphasizing the importance of interface selection in preference-based robot learning. The dataset is available to support future research.