# Extended Reality for Enhanced Human-Robot Collaboration: a Human-in-the-Loop Approach

arXiv: https://arxiv.org/abs/2403.14597 · HTML: https://mlsanity.com/paper/2403.14597/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 49 references that are in the map.
- First posted on arXiv 21 Mar 2024.

## Builds on (most cited first, 9 of 9)

- [Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation](https://mlsanity.com/paper/1710.04615/) (2017)
- [Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks](https://mlsanity.com/paper/2110.03655/) (2021)
- [Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfaces](https://mlsanity.com/paper/2203.03254/) (2022)
- [Virtual Barriers in Augmented Reality for Safe and Effective Human-Robot Cooperation in Manufacturing](https://mlsanity.com/paper/2104.05211/) (2021)
- [Task-Level Authoring for Remote Robot Teleoperation](https://mlsanity.com/paper/2109.02301/) (2021)
- [Immersive Demonstrations are the Key to Imitation Learning](https://mlsanity.com/paper/2301.09157/) (2023)
- [AR Point&Click: An Interface for Setting Robot Navigation Goals](https://mlsanity.com/paper/2203.15219/) (2022)
- [A Complementary Framework for Human-Robot Collaboration with a Mixed AR-Haptic Interface](https://mlsanity.com/paper/2210.06003/) (2022)
- [Human-Robot Skill Transfer with Enhanced Compliance via Dynamic Movement Primitives](https://mlsanity.com/paper/2304.05703/) (2023)

## Abstract (arXiv)

The rise of automation has provided an opportunity to achieve higher efficiency in manufacturing processes, yet it often compromises the flexibility required to promptly respond to evolving market needs and meet the demand for customization. Human-robot collaboration attempts to tackle these challenges by combining the strength and precision of machines with human ingenuity and perceptual understanding. In this paper, we conceptualize and propose an implementation framework for an autonomous, machine learning-based manipulator that incorporates human-in-the-loop principles and leverages Extended Reality (XR) to facilitate intuitive communication and programming between humans and robots. Furthermore, the conceptual framework foresees human involvement directly in the robot learning process, resulting in higher adaptability and task generalization. The paper highlights key technologies enabling the proposed framework, emphasizing the importance of developing the digital ecosystem as a whole. Additionally, we review the existent implementation approaches of XR in human-robot collaboration, showcasing diverse perspectives and methodologies. The challenges and future outlooks are discussed, delving into the major obstacles and potential research avenues of XR for more natural human-robot interaction and integration in the industrial landscape.