# Leveraging Deep Reinforcement Learning for Metacognitive Interventions across Intelligent Tutoring Systems

arXiv: https://arxiv.org/abs/2304.09821 · HTML: https://mlsanity.com/paper/2304.09821/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 27 references that are in the map.
- First posted on arXiv 17 Apr 2023.

## Builds on (most cited first, 7 of 7)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Investigating the Impact of Backward Strategy Learning in a Logic Tutor: Aiding Subgoal Learning towards Improved Problem Solving](https://mlsanity.com/paper/2208.04696/) (2022)
- [Metacognition and Motivation: The Role of Time-Awareness in Preparation for Future Learning](https://mlsanity.com/paper/2303.13541/) (2023)
- [Assessing Competency Using Metacognition and Motivation: The Role of Time-Awareness in Preparation for Future Learning](https://mlsanity.com/paper/2303.14609/) (2023)
- [Preparing Unprepared Students For Future Learning](https://mlsanity.com/paper/2303.11960/) (2023)
- [The Power of Nudging: Exploring Three Interventions for Metacognitive Skills Instruction across Intelligent Tutoring Systems](https://mlsanity.com/paper/2303.11965/) (2023)
- [Mixing Backward- with Forward-Chaining for Metacognitive Skill Acquisition and Transfer](https://mlsanity.com/paper/2303.12223/) (2023)

## Built on by (most cited first, 10 of 10)

- [Exploring the Potential of Metacognitive Support Agents for Human-AI Co-Creation](https://mlsanity.com/paper/2506.12879/) (2025)
- [A Generalized Apprenticeship Learning Framework for Modeling Heterogeneous Student Pedagogical Strategies](https://mlsanity.com/paper/2406.02450/) (2024)
- [Bridging Declarative, Procedural, and Conditional Metacognitive Knowledge Gap Using Deep Reinforcement Learning](https://mlsanity.com/paper/2304.11739/) (2023)
- [Adaptive Scaffolding for Cognitive Engagement in an Intelligent Tutoring System](https://mlsanity.com/paper/2602.07308/) (2026)
- [Off-Policy Selection for Initiating Human-Centric Experimental Design](https://mlsanity.com/paper/2410.20017/) (2024)
- [Enhancing tutoring systems by leveraging tailored promptings and domain knowledge with Large Language Models](https://mlsanity.com/paper/2505.02849/) (2025)
- [Personalized Student Knowledge Modeling for Future Learning Resource Prediction](https://mlsanity.com/paper/2505.14072/) (2025)
- [Learning to Optimize Feedback for One Million Students: Insights from Multi-Armed and Contextual Bandits in Large-Scale Online Tutoring](https://mlsanity.com/paper/2508.00270/) (2025)
- [Pedagogical Safety in Educational Reinforcement Learning: Formalizing and Detecting Reward Hacking in AI Tutoring Systems](https://mlsanity.com/paper/2604.04237/) (2026)
- [Access Timing as Scaffolding: A Reinforcement Learning Approach to GenAI in Education](https://mlsanity.com/paper/2605.15850/) (2026)

## Abstract (arXiv)

This work compares two approaches to provide metacognitive interventions and their impact on preparing students for future learning across Intelligent Tutoring Systems (ITSs). In two consecutive semesters, we conducted two classroom experiments: Exp. 1 used a classic artificial intelligence approach to classify students into different metacognitive groups and provide static interventions based on their classified groups. In Exp. 2, we leveraged Deep Reinforcement Learning (DRL) to provide adaptive interventions that consider the dynamic changes in the student's metacognitive levels. In both experiments, students received these interventions that taught how and when to use a backward-chaining (BC) strategy on a logic tutor that supports a default forward-chaining strategy. Six weeks later, we trained students on a probability tutor that only supports BC without interventions. Our results show that adaptive DRL-based interventions closed the metacognitive skills gap between students. In contrast, static classifier-based interventions only benefited a subset of students who knew how to use BC in advance. Additionally, our DRL agent prepared the experimental students for future learning by significantly surpassing their control peers on both ITSs.