# Quantifying Demonstration Quality for Robot Learning and Generalization

arXiv: https://arxiv.org/abs/2203.13843 · HTML: https://mlsanity.com/paper/2203.13843/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 25 Mar 2022.

## Placed in

- Offline RL and imitation learning from data / [Robot learning from demonstration](https://mlsanity.com/offlinerl/robot-learning-from-demonstration/)

## Built on by (most cited first, 4 of 4)

- [Consistency Matters: Defining Demonstration Data Quality Metrics in Robot Learning from Demonstration](https://mlsanity.com/paper/2412.14309/) (2024)
- [How Can Everyday Users Efficiently Teach Robots by Demonstrations?](https://mlsanity.com/paper/2310.13083/) (2023)
- [Instrumentation for Imitation Learning: Enhancing Training Datasets for Clothes Hanger Insertion](https://mlsanity.com/paper/2605.23847/) (2026)
- [User-Driven Learning from Demonstration: A Trajectory and Impedance Learning Method](https://mlsanity.com/paper/2607.16998/) (2026)

## Abstract (arXiv)

Learning from Demonstration (LfD) seeks to democratize robotics by enabling diverse end-users to teach robots to perform a task by providing demonstrations. However, most LfD techniques assume users provide optimal demonstrations. This is not always the case in real applications where users are likely to provide demonstrations of varying quality, that may change with expertise and other factors. Demonstration quality plays a crucial role in robot learning and generalization. Hence, it is important to quantify the quality of the provided demonstrations before using them for robot learning. In this paper, we propose quantifying the quality of the demonstrations based on how well they perform in the learned task. We hypothesize that task performance can give an indication of the generalization performance on similar tasks. The proposed approach is validated in a user study (N = 27). Users with different robotics expertise levels were recruited to teach a PR2 robot a generic task (pressing a button) under different task constraints. They taught the robot in two sessions on two different days to capture their teaching behaviour across sessions. The task performance was utilized to classify the provided demonstrations into high-quality and low-quality sets. The results show a significant Pearson correlation coefficient (R = 0.85, p < 0.0001) between the task performance and generalization performance across all participants. We also found that users clustered into two groups: Users who provided high-quality demonstrations from the first session, assigned to the fast-adapters group, and users who provided low-quality demonstrations in the first session and then improved with practice, assigned to the slow-adapters group. These results highlight the importance of quantifying demonstration quality, which can be indicative of the adaptation level of the user to the task.