# Experience-driven discovery of planning strategies

arXiv: https://arxiv.org/abs/2412.03111 · HTML: https://mlsanity.com/paper/2412.03111/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 3 of its 24 references that are in the map.
- First posted on arXiv 4 Dec 2024.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 3 of 3)

- [Learning to select computations](https://mlsanity.com/paper/1711.06892/) (2017)
- [Selecting Computations: Theory and Applications](https://mlsanity.com/paper/1408.2048/) (2014)
- [Have I done enough planning or should I plan more?](https://mlsanity.com/paper/2201.00764/) (2022)

## Built on by (most cited first, 1 of 1)

- [Individual differences in the cognitive mechanisms of planning strategy discovery](https://mlsanity.com/paper/2505.23519/) (2025)

## Abstract (arXiv)

One explanation for how people can plan efficiently despite limited cognitive resources is that we possess a set of adaptive planning strategies and know when and how to use them. But how are these strategies acquired? While previous research has studied how individuals learn to choose among existing strategies, little is known about the process of forming new planning strategies. In this work, we propose that new planning strategies are discovered through metacognitive reinforcement learning. To test this, we designed a novel experiment to investigate the discovery of new planning strategies. We then present metacognitive reinforcement learning models and demonstrate their capability for strategy discovery as well as show that they provide a better explanation of human strategy discovery than alternative learning mechanisms. However, when fitted to human data, these models exhibit a slower discovery rate than humans, leaving room for improvement.