# EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning

arXiv: https://arxiv.org/abs/1807.08364 · HTML: https://mlsanity.com/paper/1807.08364/

## Key facts

- Cited by 100 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 9 of its 29 references that are in the map.
- First posted on arXiv 22 Jul 2018.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 9 of 9)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [Query-Efficient Imitation Learning for End-to-End Autonomous Driving](https://mlsanity.com/paper/1605.06450/) (2016)
- [Safe Learning of Regions of Attraction for Uncertain, Nonlinear Systems with Gaussian Processes](https://mlsanity.com/paper/1603.04915/) (2016)

## Built on by (most cited first, 30 of 100)

- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [HG-DAgger: Interactive Imitation Learning with Human Experts](https://mlsanity.com/paper/1810.02890/) (2018)
- [RT-H: Action Hierarchies Using Language](https://mlsanity.com/paper/2403.01823/) (2024)
- [A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges](https://mlsanity.com/paper/2309.02473/) (2023)
- [ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning](https://mlsanity.com/paper/2109.08273/) (2021)
- [Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment](https://mlsanity.com/paper/2211.08416/) (2022)
- [Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences](https://mlsanity.com/paper/2002.09089/) (2020)
- [Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation](https://mlsanity.com/paper/2112.05251/) (2021)
- [NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesis](https://mlsanity.com/paper/2301.08556/) (2023)
- [Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Sampling](https://mlsanity.com/paper/2408.17355/) (2024)
- [LazyDAgger: Reducing Context Switching in Interactive Imitation Learning](https://mlsanity.com/paper/2104.00053/) (2021)
- [Failure Prediction at Runtime for Generative Robot Policies](https://mlsanity.com/paper/2510.09459/) (2025)
- [On the Sample Complexity of Stability Constrained Imitation Learning](https://mlsanity.com/paper/2102.09161/) (2021)
- [Model-Based Runtime Monitoring with Interactive Imitation Learning](https://mlsanity.com/paper/2310.17552/) (2023)
- [Multi-Task Interactive Robot Fleet Learning with Visual World Models](https://mlsanity.com/paper/2410.22689/) (2024)
- [Diff-DAgger: Uncertainty Estimation with Diffusion Policy for Robotic Manipulation](https://mlsanity.com/paper/2410.14868/) (2024)
- [Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervision](https://mlsanity.com/paper/2206.14349/) (2022)
- [RLIF: Interactive Imitation Learning as Reinforcement Learning](https://mlsanity.com/paper/2311.12996/) (2023)
- [Human-in-the-Loop Task and Motion Planning for Imitation Learning](https://mlsanity.com/paper/2310.16014/) (2023)
- [Eliciting Compatible Demonstrations for Multi-Human Imitation Learning](https://mlsanity.com/paper/2210.08073/) (2022)
- [When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learning](https://mlsanity.com/paper/2210.10765/) (2022)
- [Safe Reinforcement Learning with Chance-constrained Model Predictive Control](https://mlsanity.com/paper/2112.13941/) (2021)
- [When to Trust Imagination: Adaptive Action Execution for World Action Models](https://mlsanity.com/paper/2605.06222/) (2026)
- [PATO: Policy Assisted TeleOperation for Scalable Robot Data Collection](https://mlsanity.com/paper/2212.04708/) (2022)
- [Learning from Active Human Involvement through Proxy Value Propagation](https://mlsanity.com/paper/2502.03369/) (2025)
- [MEGA-DAgger: Imitation Learning with Multiple Imperfect Experts](https://mlsanity.com/paper/2303.00638/) (2023)
- [Safety-aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Driving](https://mlsanity.com/paper/2311.10747/) (2023)
- [So You Think You Can Scale Up Autonomous Robot Data Collection?](https://mlsanity.com/paper/2411.01813/) (2024)
- [State Dropout-Based Curriculum Reinforcement Learning for Self-Driving at Unsignalized Intersections](https://mlsanity.com/paper/2207.04361/) (2022)
- [Asking for Help: Failure Prediction in Behavioral Cloning through Value Approximation](https://mlsanity.com/paper/2302.04334/) (2023)

## Abstract (arXiv)

While imitation learning is often used in robotics, the approach frequently suffers from data mismatch and compounding errors. DAgger is an iterative algorithm that addresses these issues by aggregating training data from both the expert and novice policies, but does not consider the impact of safety. We present a probabilistic extension to DAgger, which attempts to quantify the confidence of the novice policy as a proxy for safety. Our method, EnsembleDAgger, approximates a Gaussian Process using an ensemble of neural networks. Using the variance as a measure of confidence, we compute a decision rule that captures how much we doubt the novice, thus determining when it is safe to allow the novice to act. With this approach, we aim to maximize the novice's share of actions, while constraining the probability of failure. We demonstrate improved safety and learning performance compared to other DAgger variants and classic imitation learning on an inverted pendulum and in the MuJoCo HalfCheetah environment.