# Interactive Robot Training for Non-Markov Tasks

arXiv: https://arxiv.org/abs/2003.02232 · HTML: https://mlsanity.com/paper/2003.02232/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2020.

## Built on by (most cited first, 9 of 9)

- [Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments](https://mlsanity.com/paper/2302.11649/) (2023)
- [Learning Multimodal Rewards from Rankings](https://mlsanity.com/paper/2109.12750/) (2021)
- [Data-Efficient Learning of Natural Language to Linear Temporal Logic Translators for Robot Task Specification](https://mlsanity.com/paper/2303.08006/) (2023)
- [Learning Reward Functions from Scale Feedback](https://mlsanity.com/paper/2110.00284/) (2021)
- [APReL: A Library for Active Preference-based Reward Learning Algorithms](https://mlsanity.com/paper/2108.07259/) (2021)
- [Non-Markovian Reward Modelling from Trajectory Labels via Interpretable Multiple Instance Learning](https://mlsanity.com/paper/2205.15367/) (2022)
- [Batch Active Learning of Reward Functions from Human Preferences](https://mlsanity.com/paper/2402.15757/) (2024)
- [Learning Preferences for Interactive Autonomy](https://mlsanity.com/paper/2210.10899/) (2022)
- [Explaining Reward Functions to Humans for Better Human-Robot Collaboration](https://mlsanity.com/paper/2110.04192/) (2021)

## Abstract (arXiv)

Defining sound and complete specifications for robots using formal languages is challenging, while learning formal specifications directly from demonstrations can lead to over-constrained task policies. In this paper, we propose a Bayesian interactive robot training framework that allows the robot to learn from both demonstrations provided by a teacher, and that teacher's assessments of the robot's task executions. We also present an active learning approach -- inspired by uncertainty sampling -- to identify the task execution with the most uncertain degree of acceptability. Through a simulated experiment, we demonstrate that our active learning approach identifies a teacher's intended task specification with an equivalent or greater similarity when compared to an approach that learns purely from demonstrations. Finally, we demonstrate the efficacy of our approach in a real-world setting through a user-study based on teaching a robot to set a dinner table.