# Neural-encoding Human Experts' Domain Knowledge to Warm Start Reinforcement Learning

arXiv: https://arxiv.org/abs/1902.06007 · HTML: https://mlsanity.com/paper/1902.06007/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2019.

## Built on by (most cited first, 7 of 7)

- [A Survey on Interpretable Reinforcement Learning](https://mlsanity.com/paper/2112.13112/) (2021)
- [Learning Interpretable, High-Performing Policies for Autonomous Driving](https://mlsanity.com/paper/2202.02352/) (2022)
- [Natural Language Specification of Reinforcement Learning Policies through Differentiable Decision Trees](https://mlsanity.com/paper/2101.07140/) (2021)
- [Towards Reconciling Usability and Usefulness of Explainable AI Methodologies](https://mlsanity.com/paper/2301.05347/) (2023)
- [Using Cognitive Models to Train Warm Start Reinforcement Learning Agents for Human-Computer Interactions](https://mlsanity.com/paper/2103.06160/) (2021)
- [Ask-AC: An Initiative Advisor-in-the-Loop Actor-Critic Framework](https://mlsanity.com/paper/2207.01955/) (2022)
- [Towards Fully Automated Decision-Making Systems for Greenhouse Control: Challenges and Opportunities](https://mlsanity.com/paper/2503.21640/) (2025)

## Abstract (arXiv)

Deep reinforcement learning has been successful in a variety of tasks, such as game playing and robotic manipulation. However, attempting to learn textittabula rasa disregards the logical structure of many domains as well as the wealth of readily available knowledge from domain experts that could help "warm start" the learning process. We present a novel reinforcement learning technique that allows for intelligent initialization of a neural network weights and architecture. Our approach permits the encoding domain knowledge directly into a neural decision tree, and improves upon that knowledge with policy gradient updates. We empirically validate our approach on two OpenAI Gym tasks and two modified StarCraft 2 tasks, showing that our novel architecture outperforms multilayer-perceptron and recurrent architectures. Our knowledge-based framework finds superior policies compared to imitation learning-based and prior knowledge-based approaches. Importantly, we demonstrate that our approach can be used by untrained humans to initially provide >80% increase in expected reward relative to baselines prior to training (p < 0.001), which results in a >60% increase in expected reward after policy optimization (p = 0.011).