# Pre-training Neural Networks with Human Demonstrations for Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1709.04083 · HTML: https://mlsanity.com/paper/1709.04083/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Sep 2017.

## Built on by (most cited first, 15 of 15)

- [The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors](https://mlsanity.com/paper/2101.11071/) (2021)
- [Distilling Motion Planner Augmented Policies into Visual Control Policies for Robot Manipulation](https://mlsanity.com/paper/2111.06383/) (2021)
- [Tackling Vision Language Tasks Through Learning Inner Monologues](https://mlsanity.com/paper/2308.09970/) (2023)
- [Multi-Agent Transfer Learning in Reinforcement Learning-Based Ride-Sharing Systems](https://mlsanity.com/paper/2112.00424/) (2021)
- [Learning from Ambiguous Demonstrations with Self-Explanation Guided Reinforcement Learning](https://mlsanity.com/paper/2110.05286/) (2021)
- [A novel policy for pre-trained Deep Reinforcement Learning for Speech Emotion Recognition](https://mlsanity.com/paper/2101.00738/) (2021)
- [Transfer Learning and Curriculum Learning in Sokoban](https://mlsanity.com/paper/2105.11702/) (2021)
- [Pretraining & Reinforcement Learning: Sharpening the Axe Before Cutting the Tree](https://mlsanity.com/paper/2110.02497/) (2021)
- [Accelerating Self-Imitation Learning from Demonstrations via Policy Constraints and Q-Ensemble](https://mlsanity.com/paper/2212.03562/) (2022)
- [Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2303.01170/) (2023)
- [Efficiently Training On-Policy Actor-Critic Networks in Robotic Deep Reinforcement Learning with Demonstration-like Sampled Exploration](https://mlsanity.com/paper/2109.13005/) (2021)
- [Efficient Reinforcement Learning from Demonstration Using Local Ensemble and Reparameterization with Split and Merge of Expert Policies](https://mlsanity.com/paper/2205.11019/) (2022)
- [Policy Optimization with Smooth Guidance Learned from State-Only Demonstrations](https://mlsanity.com/paper/2401.00162/) (2023)
- [Generalising Multi-Agent Cooperation through Task-Agnostic Communication](https://mlsanity.com/paper/2403.06750/) (2024)
- [Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2501.15495/) (2025)

## Abstract (arXiv)

Deep reinforcement learning (deep RL) has achieved superior performance in complex sequential tasks by using a deep neural network as its function approximator and by learning directly from raw images. A drawback of using raw images is that deep RL must learn the state feature representation from the raw images in addition to learning a policy. As a result, deep RL can require a prohibitively large amount of training time and data to reach reasonable performance, making it difficult to use deep RL in real-world applications, especially when data is expensive. In this work, we speed up training by addressing half of what deep RL is trying to solve --- learning features. Our approach is to learn some of the important features by pre-training deep RL network's hidden layers via supervised learning using a small set of human demonstrations. We empirically evaluate our approach using deep Q-network (DQN) and asynchronous advantage actor-critic (A3C) algorithms on the Atari 2600 games of Pong, Freeway, and Beamrider. Our results show that: 1) pre-training with human demonstrations in a supervised learning manner is better at discovering features relative to pre-training naively in DQN, and 2) initializing a deep RL network with a pre-trained model provides a significant improvement in training time even when pre-training from a small number of human demonstrations.