# How to Train Your Robots? The Impact of Demonstration Modality on Imitation Learning

arXiv: https://arxiv.org/abs/2503.07017 · HTML: https://mlsanity.com/paper/2503.07017/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 18 of its 32 references that are in the map.
- First posted on arXiv 10 Mar 2025.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots](https://mlsanity.com/paper/2402.10329/) (2024)
- [RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation](https://mlsanity.com/paper/1811.02790/) (2018)
- [AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System](https://mlsanity.com/paper/2307.04577/) (2023)
- [Visual Imitation Made Easy](https://mlsanity.com/paper/2008.04899/) (2020)
- [Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment](https://mlsanity.com/paper/2211.08416/) (2022)
- [Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation](https://mlsanity.com/paper/2109.13841/) (2021)
- [Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning](https://mlsanity.com/paper/2407.03162/) (2024)
- [Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation](https://mlsanity.com/paper/1810.07121/) (2018)
- [Data Quality in Imitation Learning](https://mlsanity.com/paper/2306.02437/) (2023)
- [Re-Mix: Optimizing Data Mixtures for Large Scale Imitation Learning](https://mlsanity.com/paper/2408.14037/) (2024)
- [Efficient Data Collection for Robotic Manipulation via Compositional Generalization](https://mlsanity.com/paper/2403.05110/) (2024)
- [AR2-D2:Training a Robot Without a Robot](https://mlsanity.com/paper/2306.13818/) (2023)
- [Eliciting Compatible Demonstrations for Multi-Human Imitation Learning](https://mlsanity.com/paper/2210.08073/) (2022)
- [A VR System for Immersive Teleoperation and Live Exploration with a Mobile Robot](https://mlsanity.com/paper/1908.02949/) (2019)

## Built on by (most cited first, 11 of 11)

- [RaC: Robot Learning for Long-Horizon Tasks by Scaling Recovery and Correction](https://mlsanity.com/paper/2509.07953/) (2025)
- [Is Diversity All You Need for Scalable Robotic Manipulation?](https://mlsanity.com/paper/2507.06219/) (2025)
- [Dexora: Open-source VLA for High-DoF Bimanual Dexterity](https://mlsanity.com/paper/2605.18722/) (2026)
- [XRoboToolkit: A Cross-Platform Framework for Robot Teleoperation](https://mlsanity.com/paper/2508.00097/) (2025)
- [Learning from the Best: Smoothness-Driven Metrics for Data Quality in Imitation Learning](https://mlsanity.com/paper/2604.23000/) (2026)
- [Ark: An Open-source Python-based Framework for Robot Learning](https://mlsanity.com/paper/2506.21628/) (2025)
- [EasyMimic: A Low-Cost Framework for Robot Imitation Learning from Human Videos](https://mlsanity.com/paper/2602.11464/) (2026)
- [Behavioral Mode Discovery for Fine-tuning Multimodal Generative Policies](https://mlsanity.com/paper/2605.11387/) (2026)
- [Reinforcement Learning-Guided Retrieval with Soft Fusion for Robust Multimodal Imitation Learning under Missing Modalities](https://mlsanity.com/paper/2606.15514/) (2026)
- [SPACE: Enabling Learning from Cross-Robot Data Toward Generalist Policies](https://mlsanity.com/paper/2606.24049/) (2026)
- [Imperfection for Precision: Upcycling Imperfect Data for High-Precision Robotic Manipulation](https://mlsanity.com/paper/2609.26672/) (2026)

## Abstract (arXiv)

Imitation learning is a promising approach for learning robot policies with user-provided data. The way demonstrations are provided, i.e., demonstration modality, influences the quality of the data. While existing research shows that kinesthetic teaching (physically guiding the robot) is preferred by users for the intuitiveness and ease of use, the majority of existing manipulation datasets were collected through teleoperation via a VR controller or spacemouse. In this work, we investigate how different demonstration modalities impact downstream learning performance as well as user experience. Specifically, we compare low-cost demonstration modalities including kinesthetic teaching, teleoperation with a VR controller, and teleoperation with a spacemouse controller. We experiment with three table-top manipulation tasks with different motion constraints. We evaluate and compare imitation learning performance using data from different demonstration modalities, and collected subjective feedback on user experience. Our results show that kinesthetic teaching is rated the most intuitive for controlling the robot and provides cleanest data for best downstream learning performance. However, it is not preferred as the way for large-scale data collection due to the physical load. Based on such insight, we propose a simple data collection scheme that relies on a small number of kinesthetic demonstrations mixed with data collected through teleoperation to achieve the best overall learning performance while maintaining low data-collection effort.