# Exploratory Gradient Boosting for Reinforcement Learning in Complex Domains

arXiv: https://arxiv.org/abs/1603.04119 · HTML: https://mlsanity.com/paper/1603.04119/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Mar 2016.

## Built on by (most cited first, 8 of 8)

- [Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos](https://mlsanity.com/paper/2206.11795/) (2022)
- [Exploration in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2205.00824/) (2022)
- [AndroidEnv: A Reinforcement Learning Platform for Android](https://mlsanity.com/paper/2105.13231/) (2021)
- [Deep Reinforcement Learning for Autonomous Cyber Defence: A Survey](https://mlsanity.com/paper/2310.07745/) (2023)
- [Boosted Off-Policy Learning](https://mlsanity.com/paper/2208.01148/) (2022)
- [Krylov-Bellman boosting: Super-linear policy evaluation in general state spaces](https://mlsanity.com/paper/2210.11377/) (2022)
- [Gradient Boosting Reinforcement Learning](https://mlsanity.com/paper/2407.08250/) (2024)
- [Local Updates, Global Learning (LUGL): Playing Games with non-incremental Learners](https://mlsanity.com/paper/2609.03660/) (2026)

## Abstract (arXiv)

High-dimensional observations and complex real-world dynamics present major challenges in reinforcement learning for both function approximation and exploration. We address both of these challenges with two complementary techniques: First, we develop a gradient-boosting style, non-parametric function approximator for learning on Q-function residuals. And second, we propose an exploration strategy inspired by the principles of state abstraction and information acquisition under uncertainty. We demonstrate the empirical effectiveness of these techniques, first, as a preliminary check, on two standard tasks (Blackjack and n-Chain), and then on two much larger and more realistic tasks with high-dimensional observation spaces. Specifically, we introduce two benchmarks built within the game Minecraft where the observations are pixel arrays of the agent's visual field. A combination of our two algorithmic techniques performs competitively on the standard reinforcement-learning tasks while consistently and substantially outperforming baselines on the two tasks with high-dimensional observation spaces. The new function approximator, exploration strategy, and evaluation benchmarks are each of independent interest in the pursuit of reinforcement-learning methods that scale to real-world domains.