# Pre-training with Non-expert Human Demonstration for Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1812.08904 · HTML: https://mlsanity.com/paper/1812.08904/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2018.

## Built on by (most cited first, 8 of 8)

- [From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by Composing Old Ones](https://mlsanity.com/paper/2509.25123/) (2025)
- [Minimizing Human Assistance: Augmenting a Single Demonstration for Deep Reinforcement Learning](https://mlsanity.com/paper/2209.11275/) (2022)
- [Improving Reinforcement Learning with Human Assistance: An Argument for Human Subject Studies with HIPPO Gym](https://mlsanity.com/paper/2102.02639/) (2021)
- [Co-Imitation Learning without Expert Demonstration](https://mlsanity.com/paper/2103.14823/) (2021)
- [Pretraining & Reinforcement Learning: Sharpening the Axe Before Cutting the Tree](https://mlsanity.com/paper/2110.02497/) (2021)
- [Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2303.01170/) (2023)
- [Exploring Adaptive MCTS with TD Learning in miniXCOM](https://mlsanity.com/paper/2210.05014/) (2022)
- [Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2501.15495/) (2025)

## Abstract (arXiv)

Deep reinforcement learning (deep RL) has achieved superior performance in complex sequential tasks by using deep neural networks as function approximators to learn directly from raw input images. However, learning directly from raw images is data inefficient. The agent must learn feature representation of complex states in addition to learning a policy. As a result, deep RL typically suffers from slow learning speeds and often requires a prohibitively large amount of training time and data to reach reasonable performance, making it inapplicable to real-world settings where data is expensive. In this work, we improve data efficiency in deep RL by addressing one of the two learning goals, feature learning. We leverage supervised learning to pre-train on a small set of non-expert human demonstrations and empirically evaluate our approach using the asynchronous advantage actor-critic algorithms (A3C) in the Atari domain. Our results show significant improvements in learning speed, even when the provided demonstration is noisy and of low quality.