# Continuous Reinforcement Learning-based Dynamic Difficulty Adjustment in a Visual Working Memory Game

arXiv: https://arxiv.org/abs/2308.12726 · HTML: https://mlsanity.com/paper/2308.12726/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 31 references that are in the map.
- First posted on arXiv 24 Aug 2023.

## Builds on (most cited first, 2 of 2)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Dynamic Difficulty Adjustment in Virtual Reality Exergames through Experience-driven Procedural Content Generation](https://mlsanity.com/paper/2108.08762/) (2021)

## Built on by (most cited first, 2 of 2)

- [Design Process of a Self Adaptive Smart Serious Games Ecosystem](https://mlsanity.com/paper/2510.04615/) (2025)
- [OpenBlock: Constructive and Verified Content Generation for Adaptive Tile-Matching Games](https://mlsanity.com/paper/2609.22177/) (2026)

## Abstract (arXiv)

Dynamic Difficulty Adjustment (DDA) is a viable approach to enhance a player's experience in video games. Recently, Reinforcement Learning (RL) methods have been employed for DDA in non-competitive games; nevertheless, they rely solely on discrete state-action space with a small search space. In this paper, we propose a continuous RL-based DDA methodology for a visual working memory (VWM) game to handle the complex search space for the difficulty of memorization. The proposed RL-based DDA tailors game difficulty based on the player's score and game difficulty in the last trial. We defined a continuous metric for the difficulty of memorization. Then, we consider the task difficulty and the vector of difficulty-score as the RL's action and state, respectively. We evaluated the proposed method through a within-subject experiment involving 52 subjects. The proposed approach was compared with two rule-based difficulty adjustment methods in terms of player's score and game experience measured by a questionnaire. The proposed RL-based approach resulted in a significantly better game experience in terms of competence, tension, and negative and positive affect. Players also achieved higher scores and win rates. Furthermore, the proposed RL-based DDA led to a significantly less decline in the score in a 20-trial session.