# Reinforcement Learning Methods for Wordle: A POMDP/Adaptive Control Approach

arXiv: https://arxiv.org/abs/2211.10298 · HTML: https://mlsanity.com/paper/2211.10298/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 32 references that are in the map.
- First posted on arXiv 15 Nov 2022.

## Builds on (most cited first, 5 of 5)

- [Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control](https://mlsanity.com/paper/2108.10315/) (2021)
- [Multiple-Environment Markov Decision Processes](https://mlsanity.com/paper/1405.4733/) (2014)
- [Percentile Queries in Multi-Dimensional Markov Decision Processes](https://mlsanity.com/paper/1410.4801/) (2014)
- [Variations on the Stochastic Shortest Path Problem](https://mlsanity.com/paper/1411.0835/) (2014)
- [Selecting Seed Words for Wordle using Character Statistics](https://mlsanity.com/paper/2202.03457/) (2022)

## Built on by (most cited first, 4 of 4)

- [Active Uncertainty Reduction for Safe and Efficient Interaction Planning: A Shielding-Aware Dual Control Approach](https://mlsanity.com/paper/2302.00171/) (2023)
- [Rollout Algorithms and Approximate Dynamic Programming for Bayesian Optimization and Sequential Estimation](https://mlsanity.com/paper/2212.07998/) (2022)
- [Optimal and Efficient Binary Questioning for Human-in-the-Loop Annotation](https://mlsanity.com/paper/2307.01578/) (2023)
- [An Approximate Dynamic Programming Framework for Occlusion-Robust Multi-Object Tracking](https://mlsanity.com/paper/2405.15137/) (2024)

## Abstract (arXiv)

In this paper we address the solution of the popular Wordle puzzle, using new reinforcement learning methods, which apply more generally to adaptive control of dynamic systems and to classes of Partially Observable Markov Decision Process (POMDP) problems. These methods are based on approximation in value space and the rollout approach, admit a straightforward implementation, and provide improved performance over various heuristic approaches. For the Wordle puzzle, they yield on-line solution strategies that are very close to optimal at relatively modest computational cost. Our methods are viable for more complex versions of Wordle and related search problems, for which an optimal strategy would be impossible to compute. They are also applicable to a wide range of adaptive sequential decision problems that involve an unknown or frequently changing environment whose parameters are estimated on-line.