# A VR System for Immersive Teleoperation and Live Exploration with a Mobile Robot

arXiv: https://arxiv.org/abs/1908.02949 · HTML: https://mlsanity.com/paper/1908.02949/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Aug 2019.

## Built on by (most cited first, 10 of 10)

- [SwinGS: Sliding Window Gaussian Splatting for Volumetric Video Streaming with Arbitrary Length](https://mlsanity.com/paper/2409.07759/) (2024)
- [How to Train Your Robots? The Impact of Demonstration Modality on Imitation Learning](https://mlsanity.com/paper/2503.07017/) (2025)
- [Generative Adversarial Super-Resolution at the Edge with Knowledge Distillation](https://mlsanity.com/paper/2209.03355/) (2022)
- [Interactive Navigation in Environments with Traversable Obstacles Using Large Language and Vision-Language Models](https://mlsanity.com/paper/2310.08873/) (2023)
- [Virtual, Augmented, and Mixed Reality for Human-Robot Interaction: A Survey and Virtual Design Element Taxonomy](https://mlsanity.com/paper/2202.11249/) (2022)
- [STDR: Spatio-Temporal Decoupling for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2505.22400/) (2025)
- [Efficient 3D Reconstruction, Streaming and Visualization of Static and Dynamic Scene Parts for Multi-client Live-telepresence in Large-scale Environments](https://mlsanity.com/paper/2211.14310/) (2022)
- [A Review of Scene Representations for Robot Manipulators](https://mlsanity.com/paper/2301.11275/) (2022)
- [Effective Virtual Reality Teleoperation of an Upper-body Humanoid with Modified Task Jacobians and Relaxed Barrier Functions for Self-Collision Avoidance](https://mlsanity.com/paper/2411.07534/) (2024)
- [Extended Reality as a Mediation Layer for Situated Human Control in Human-Robot Teaming](https://mlsanity.com/paper/2607.25047/) (2026)

## Abstract (arXiv)

Applications like disaster management and industrial inspection often require experts to enter contaminated places. To circumvent the need for physical presence, it is desirable to generate a fully immersive individual live teleoperation experience. However, standard video-based approaches suffer from a limited degree of immersion and situation awareness due to the restriction to the camera view, which impacts the navigation. In this paper, we present a novel VR-based practical system for immersive robot teleoperation and scene exploration. While being operated through the scene, a robot captures RGB-D data that is streamed to a SLAM-based live multi-client telepresence system. Here, a global 3D model of the already captured scene parts is reconstructed and streamed to the individual remote user clients where the rendering for e.g. head-mounted display devices (HMDs) is performed. We introduce a novel lightweight robot client component which transmits robot-specific data and enables a quick integration into existing robotic systems. This way, in contrast to first-person exploration systems, the operators can explore and navigate in the remote site completely independent of the current position and view of the capturing robot, complementing traditional input devices for teleoperation. We provide a proof-of-concept implementation and demonstrate the capabilities as well as the performance of our system regarding interactive object measurements and bandwidth-efficient data streaming and visualization. Furthermore, we show its benefits over purely video-based teleoperation in a user study revealing a higher degree of situation awareness and a more precise navigation in challenging environments.