# Efficiently Combining Human Demonstrations and Interventions for Safe Training of Autonomous Systems in Real-Time

arXiv: https://arxiv.org/abs/1810.11545 · HTML: https://mlsanity.com/paper/1810.11545/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 26 Oct 2018.

## Placed in

- Offline RL and imitation learning from data / [Robot learning from demonstration](https://mlsanity.com/offlinerl/robot-learning-from-demonstration/)

## Built on by (most cited first, 8 of 8)

- [Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Tasks](https://mlsanity.com/paper/2107.05825/) (2021)
- [On games and simulators as a platform for development of artificial intelligence for command and control](https://mlsanity.com/paper/2110.11305/) (2021)
- [Combining Learning from Human Feedback and Knowledge Engineering to Solve Hierarchical Tasks in Minecraft](https://mlsanity.com/paper/2112.03482/) (2021)
- [Combining RL and IL using a dynamic, performance-based modulation over learning signals and its application to local planning](https://mlsanity.com/paper/2405.09760/) (2024)
- [BCR-DRL: Behavior- and Context-aware Reward for Deep Reinforcement Learning in Human-AI Coordination](https://mlsanity.com/paper/2408.07877/) (2024)
- [Scalable Interactive Machine Learning for Future Command and Control](https://mlsanity.com/paper/2402.06501/) (2024)
- [An Emergency Disposal Decision-making Method with Human--Machine Collaboration](https://mlsanity.com/paper/2305.18165/) (2023)
- [Learning Safe Agent Behaviour from Human Preferences and Justifications via World Models](https://mlsanity.com/paper/2607.13172/) (2026)

## Abstract (arXiv)

This paper investigates how to utilize different forms of human interaction to safely train autonomous systems in real-time by learning from both human demonstrations and interventions. We implement two components of the Cycle-of-Learning for Autonomous Systems, which is our framework for combining multiple modalities of human interaction. The current effort employs human demonstrations to teach a desired behavior via imitation learning, then leverages intervention data to correct for undesired behaviors produced by the imitation learner to teach novel tasks to an autonomous agent safely, after only minutes of training. We demonstrate this method in an autonomous perching task using a quadrotor with continuous roll, pitch, yaw, and throttle commands and imagery captured from a downward-facing camera in a high-fidelity simulated environment. Our method improves task completion performance for the same amount of human interaction when compared to learning from demonstrations alone, while also requiring on average 32% less data to achieve that performance. This provides evidence that combining multiple modes of human interaction can increase both the training speed and overall performance of policies for autonomous systems.