# Effective Virtual Reality Teleoperation of an Upper-body Humanoid with Modified Task Jacobians and Relaxed Barrier Functions for Self-Collision Avoidance

arXiv: https://arxiv.org/abs/2411.07534 · HTML: https://mlsanity.com/paper/2411.07534/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 30 references that are in the map.
- First posted on arXiv 12 Nov 2024.

## Builds on (most cited first, 12 of 12)

- [Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation](https://mlsanity.com/paper/1710.04615/) (2017)
- [Human-in-the-Loop Imitation Learning using Remote Teleoperation](https://mlsanity.com/paper/2012.06733/) (2020)
- [Feedback MPC for Torque-Controlled Legged Robots](https://mlsanity.com/paper/1905.06144/) (2019)
- [Feedback Control of a Cassie Bipedal Robot: Walking, Standing, and Riding a Segway](https://mlsanity.com/paper/1809.07279/) (2018)
- [Whole-Body Geometric Retargeting for Humanoid Robots](https://mlsanity.com/paper/1909.10080/) (2019)
- [A Collision-Free MPC for Whole-Body Dynamic Locomotion and Manipulation](https://mlsanity.com/paper/2202.12385/) (2022)
- [A VR System for Immersive Teleoperation and Live Exploration with a Mobile Robot](https://mlsanity.com/paper/1908.02949/) (2019)
- [Relaxed Logarithmic Barrier Function Based Model Predictive Control of Linear Systems](https://mlsanity.com/paper/1503.03314/) (2015)
- [Dynamic Locomotion For Passive-Ankle Biped Robots And Humanoids Using Whole-Body Locomotion Control](https://mlsanity.com/paper/1901.08100/) (2019)
- [Telexistence and Teleoperation for Walking Humanoid Robots](https://mlsanity.com/paper/1809.01578/) (2018)
- [Humanoid Self-Collision Avoidance Using Whole-Body Control with Control Barrier Functions](https://mlsanity.com/paper/2207.00692/) (2022)
- [Deploying the NASA Valkyrie Humanoid for IED Response: An Initial Approach and Evaluation Summary](https://mlsanity.com/paper/1910.00761/) (2019)

## Abstract (arXiv)

We present an approach for retartgeting off-the-shelf Virtual Reality (VR) trackers to effectively teleoperate an upper-body humanoid while ensuring self-collision-free motions. Key to the effectiveness was the proper assignment of trackers to joint sets via modified task Jacobians and relaxed barrier functions for self-collision avoidance. The approach was validated on Apptronik's Astro hardware by demonstrating manipulation capabilities on a table-top environment with pick-and-place box packing and a two-handed box pick up and handover task.