# Multi-Armed Bandits for Intelligent Tutoring Systems

arXiv: https://arxiv.org/abs/1310.3174 · HTML: https://mlsanity.com/paper/1310.3174/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2013.

## Built on by (most cited first, 21 of 21)

- [Peeking with PEAK: Sequential, Nonparametric Composite Hypothesis Tests for Means of Multiple Data Streams](https://mlsanity.com/paper/2402.06122/) (2024)
- [MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spaces](https://mlsanity.com/paper/2502.07709/) (2025)
- [Challenges in Statistical Analysis of Data Collected by a Bandit Algorithm: An Empirical Exploration in Applications to Adaptively Randomized Experiments](https://mlsanity.com/paper/2103.12198/) (2021)
- [p-Mean Regret for Stochastic Bandits](https://mlsanity.com/paper/2412.10751/) (2024)
- [Improving Artificial Teachers by Considering How People Learn and Forget](https://mlsanity.com/paper/2102.04174/) (2021)
- [Dealing with Expert Bias in Collective Decision-Making](https://mlsanity.com/paper/2106.13539/) (2021)
- [Shared Autonomy for Proximal Teaching](https://mlsanity.com/paper/2502.19899/) (2025)
- [Towards Scalable Adaptive Learning with Graph Neural Networks and Reinforcement Learning](https://mlsanity.com/paper/2305.06398/) (2023)
- [DP-NCB: Privacy Preserving Fair Bandits](https://mlsanity.com/paper/2508.03836/) (2025)
- [Reinforcement Teaching](https://mlsanity.com/paper/2204.11897/) (2022)
- [Raising Student Completion Rates with Adaptive Curriculum and Contextual Bandits](https://mlsanity.com/paper/2207.14003/) (2022)
- [Getting too personal(ized): The importance of feature choice in online adaptive algorithms](https://mlsanity.com/paper/2309.02856/) (2023)
- [Improved Performances and Motivation in Intelligent Tutoring Systems: Combining Machine Learning and Learner Choice](https://mlsanity.com/paper/2402.01669/) (2024)
- [Hierarchical Multi-Armed Bandits for the Concurrent Intelligent Tutoring of Concepts and Problems of Varying Difficulty Levels](https://mlsanity.com/paper/2408.07208/) (2024)
- [CANINE: Coaching Visually Impaired Users for Interactive Navigation with a Robot Guide Dog](https://mlsanity.com/paper/2605.19501/) (2026)
- [Prerequisite Structure Discovery in Intelligent Tutoring Systems](https://mlsanity.com/paper/2402.01672/) (2024)
- [A Pre-Trained Graph-Based Model for Adaptive Sequencing of Educational Documents](https://mlsanity.com/paper/2411.11520/) (2024)
- [A Computational Model of Inclusive Pedagogy: From Understanding to Application](https://mlsanity.com/paper/2505.02853/) (2025)
- [Learning to Allocate Resources with Censored Feedback](https://mlsanity.com/paper/2602.06565/) (2026)
- [Counterfactual learning of new adaptive instructional policies using logged data](https://mlsanity.com/paper/2606.23015/) (2026)
- [Stochasticity Is Not the Hard Part: Reduction and Complexity in Instructional Sequencing over Prerequisite DAGs](https://mlsanity.com/paper/2608.05455/) (2026)

## Abstract (arXiv)

We present an approach to Intelligent Tutoring Systems which adaptively personalizes sequences of learning activities to maximize skills acquired by students, taking into account the limited time and motivational resources. At a given point in time, the system proposes to the students the activity which makes them progress faster. We introduce two algorithms that rely on the empirical estimation of the learning progress, RiARiT that uses information about the difficulty of each exercise and ZPDES that uses much less knowledge about the problem. The system is based on the combination of three approaches. First, it leverages recent models of intrinsically motivated learning by transposing them to active teaching, relying on empirical estimation of learning progress provided by specific activities to particular students. Second, it uses state-of-the-art Multi-Arm Bandit (MAB) techniques to efficiently manage the exploration/exploitation challenge of this optimization process. Third, it leverages expert knowledge to constrain and bootstrap initial exploration of the MAB, while requiring only coarse guidance information of the expert and allowing the system to deal with didactic gaps in its knowledge. The system is evaluated in a scenario where 7-8 year old schoolchildren learn how to decompose numbers while manipulating money. Systematic experiments are presented with simulated students, followed by results of a user study across a population of 400 school children.