# A Framework for Learning from Demonstration with Minimal Human Effort

arXiv: https://arxiv.org/abs/2306.09211 · HTML: https://mlsanity.com/paper/2306.09211/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 10 of its 35 references that are in the map.
- First posted on arXiv 15 Jun 2023.

## Placed in

- Offline RL and imitation learning from data / [Robot learning from demonstration](https://mlsanity.com/offlinerl/robot-learning-from-demonstration/)

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [Overcoming Exploration in Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/1709.10089/) (2017)
- [Shared Autonomy via Deep Reinforcement Learning](https://mlsanity.com/paper/1802.01744/) (2018)
- [Interactive Policy Learning through Confidence-Based Autonomy](https://mlsanity.com/paper/1401.3439/) (2014)
- [The STRANDS Project: Long-Term Autonomy in Everyday Environments](https://mlsanity.com/paper/1604.04384/) (2016)
- [Learning Models for Shared Control of Human-Machine Systems with Unknown Dynamics](https://mlsanity.com/paper/1808.08268/) (2018)
- [Explore, Exploit or Listen: Combining Human Feedback and Policy Model to Speed up Deep Reinforcement Learning in 3D Worlds](https://mlsanity.com/paper/1709.03969/) (2017)
- [Synthesis of Shared Control Protocols with Provable Safety and Performance Guarantees](https://mlsanity.com/paper/1610.08500/) (2016)

## Built on by (most cited first, 4 of 4)

- ["Give Me an Example Like This": Episodic Active Reinforcement Learning from Demonstrations](https://mlsanity.com/paper/2406.03069/) (2024)
- [A Q-learning approach to the continuous control problem of robot inverted pendulum balancing](https://mlsanity.com/paper/2312.02649/) (2023)
- [Robot Policy Transfer with Online Demonstrations: An Active Reinforcement Learning Approach](https://mlsanity.com/paper/2503.12993/) (2025)
- [Leveraging Human-In-The-Loop Demonstrations in Reinforcement Learning for Digital Twin-Driven Robot Flexibility](https://mlsanity.com/paper/2610.12140/) (2026)

## Abstract (arXiv)

We consider robot learning in the context of shared autonomy, where control of the system can switch between a human teleoperator and autonomous control. In this setting we address reinforcement learning, and learning from demonstration, where there is a cost associated with human time. This cost represents the human time required to teleoperate the robot, or recover the robot from failures. For each episode, the agent must choose between requesting human teleoperation, or using one of its autonomous controllers. In our approach, we learn to predict the success probability for each controller, given the initial state of an episode. This is used in a contextual multi-armed bandit algorithm to choose the controller for the episode. A controller is learnt online from demonstrations and reinforcement learning so that autonomous performance improves, and the system becomes less reliant on the teleoperator with more experience. We show that our approach to controller selection reduces the human cost to perform two simulated tasks and a single real-world task.