# Q-learning with online random forests

arXiv: https://arxiv.org/abs/2204.03771 · HTML: https://mlsanity.com/paper/2204.03771/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 23 references that are in the map.
- First posted on arXiv 7 Apr 2022.

## Builds on (most cited first, 6 of 6)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [An Introduction to Deep Reinforcement Learning](https://mlsanity.com/paper/1811.12560/) (2018)
- [Temporal Difference Variational Auto-Encoder](https://mlsanity.com/paper/1806.03107/) (2018)
- [Human vs. Computer Go: Review and Prospect](https://mlsanity.com/paper/1606.02032/) (2016)

## Built on by (most cited first, 2 of 2)

- [Mitigating Information Loss in Tree-Based Reinforcement Learning via Direct Optimization](https://mlsanity.com/paper/2408.08761/) (2024)
- [Upside-Down Reinforcement Learning for More Interpretable Optimal Control](https://mlsanity.com/paper/2411.11457/) (2024)

## Abstract (arXiv)

Q-learning is the most fundamental model-free reinforcement learning algorithm. Deployment of Q-learning requires approximation of the state-action value function (also known as the Q-function). In this work, we provide online random forests as Q-function approximators and propose a novel method wherein the random forest is grown as learning proceeds (through expanding forests). We demonstrate improved performance of our methods over state-of-the-art Deep Q-Networks in two OpenAI gyms (`blackjack' and `inverted pendulum') but not in the `lunar lander' gym. We suspect that the resilience to overfitting enjoyed by random forests recommends our method for common tasks that do not require a strong representation of the problem domain. We show that expanding forests (in which the number of trees increases as data comes in) improve performance, suggesting that expanding forests are viable for other applications of online random forests beyond the reinforcement learning setting.