# Efficient Active Imitation Learning with Random Network Distillation

arXiv: https://arxiv.org/abs/2411.01894 · HTML: https://mlsanity.com/paper/2411.01894/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 8 of its 18 references that are in the map.
- First posted on arXiv 4 Nov 2024.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 8 of 8)

- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [HG-DAgger: Interactive Imitation Learning with Human Experts](https://mlsanity.com/paper/1810.02890/) (2018)
- [Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Critics](https://mlsanity.com/paper/2005.04269/) (2020)
- [Goal-conditioned Imitation Learning](https://mlsanity.com/paper/1906.05838/) (2019)
- [EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning](https://mlsanity.com/paper/1807.08364/) (2018)
- [LazyDAgger: Reducing Context Switching in Interactive Imitation Learning](https://mlsanity.com/paper/2104.00053/) (2021)
- [Imitation Learning with Concurrent Actions in 3D Games](https://mlsanity.com/paper/1803.05402/) (2018)
- [Improving Generalization in Game Agents with Data Augmentation in Imitation Learning](https://mlsanity.com/paper/2309.12815/) (2023)

## Built on by (most cited first, 7 of 7)

- [Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanism](https://mlsanity.com/paper/2506.09176/) (2025)
- [Real-Time Diffusion Policies for Games: Enhancing Consistency Policies with Q-Ensembles](https://mlsanity.com/paper/2503.16978/) (2025)
- [A Continual Offline Reinforcement Learning Benchmark for Navigation Tasks](https://mlsanity.com/paper/2506.02883/) (2025)
- [AutoIntervene: Calibrated Intervention for Action-Chunking Imitation Learning Policies](https://mlsanity.com/paper/2608.07065/) (2026)
- [TROFI: Trajectory-Ranked Offline Inverse Reinforcement Learning](https://mlsanity.com/paper/2506.22008/) (2025)
- [EAPO: Enhancing Policy Optimization with On-Demand Expert Assistance](https://mlsanity.com/paper/2509.23730/) (2025)
- [Sample-Efficient Expert Query Control in Active Imitation Learning via Conformal Prediction](https://mlsanity.com/paper/2512.00453/) (2025)

## Abstract (arXiv)

Developing agents for complex and underspecified tasks, where no clear objective exists, remains challenging but offers many opportunities. This is especially true in video games, where simulated players (bots) need to play realistically, and there is no clear reward to evaluate them. While imitation learning has shown promise in such domains, these methods often fail when agents encounter out-of-distribution scenarios during deployment. Expanding the training dataset is a common solution, but it becomes impractical or costly when relying on human demonstrations. This article addresses active imitation learning, aiming to trigger expert intervention only when necessary, reducing the need for constant expert input along training. We introduce Random Network Distillation DAgger (RND-DAgger), a new active imitation learning method that limits expert querying by using a learned state-based out-of-distribution measure to trigger interventions. This approach avoids frequent expert-agent action comparisons, thus making the expert intervene only when it is useful. We evaluate RND-DAgger against traditional imitation learning and other active approaches in 3D video games (racing and third-person navigation) and in a robotic locomotion task and show that RND-DAgger surpasses previous methods by reducing expert queries. https://sites.google.com/view/rnd-dagger