# Hierarchical Multi-Armed Bandits for the Concurrent Intelligent Tutoring of Concepts and Problems of Varying Difficulty Levels

arXiv: https://arxiv.org/abs/2408.07208 · HTML: https://mlsanity.com/paper/2408.07208/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 18 references that are in the map.
- First posted on arXiv 10 Aug 2024.

## Builds on (most cited first, 5 of 5)

- [Multi-Armed Bandits for Intelligent Tutoring Systems](https://mlsanity.com/paper/1310.3174/) (2013)
- [pyBKT: An Accessible Python Library of Bayesian Knowledge Tracing Models](https://mlsanity.com/paper/2105.00385/) (2021)
- [Combining Difficulty Ranking with Multi-Armed Bandits to Sequence Educational Content](https://mlsanity.com/paper/1804.05212/) (2018)
- [Adaptive Task Assignment in Online Learning Environments](https://mlsanity.com/paper/1606.07233/) (2016)
- [Teenagers and Artificial Intelligence: Bootcamp Experience and Lessons Learned](https://mlsanity.com/paper/2312.10067/) (2023)

## Built on by (most cited first, 1 of 1)

- [Hierarchical Dataset Selection for High-Quality Data Sharing](https://mlsanity.com/paper/2512.10952/) (2025)

## Abstract (arXiv)

Remote education has proliferated in the twenty-first century, yielding rise to intelligent tutoring systems. In particular, research has found multi-armed bandit (MAB) intelligent tutors to have notable abilities in traversing the exploration-exploitation trade-off landscape for student problem recommendations. Prior literature, however, contains a significant lack of open-sourced MAB intelligent tutors, which impedes potential applications of these educational MAB recommendation systems. In this paper, we combine recent literature on MAB intelligent tutoring techniques into an open-sourced and simply deployable hierarchical MAB algorithm, capable of progressing students concurrently through concepts and problems, determining ideal recommended problem difficulties, and assessing latent memory decay. We evaluate our algorithm using simulated groups of 500 students, utilizing Bayesian Knowledge Tracing to estimate students' content mastery. Results suggest that our algorithm, when turned difficulty-agnostic, significantly boosts student success, and that the further addition of problem-difficulty adaptation notably improves this metric.