mlsanityData to 7 Oct 2026

MC-CPO: Mastery-Conditioned Constrained Policy Optimization for Pedagogically Safe Intelligent Tutoring Systems

arXiv 2604.04251 ↗First posted 5 Apr 2026cs.AI cs.CY cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Intelligent tutoring systems increasingly rely on reinforcement learning to personalise instruction, yet optimising for observable engagement signals can systematically decouple learner activity from genuine knowledge acquisition. Analysing over 21 million student interactions across two deployed platforms, we find engagement events without corresponding mastery gains occur in 26.5% of interactions on Junyi Academy (72,758 students) and 3.1% on XES3G5M (14,453 students, NeurIPS 2023), confirming this pattern is directly observable in deployed educational technology at scale. We introduce Mastery-Conditioned Constrained Policy Optimisation (MC-CPO), a reinforcement learning framework that addresses this problem structurally. MC-CPO conditions the admissible instructional action space on learner mastery state: a concept becomes available only when prerequisite knowledge meets a mastery threshold, yielding an action space that expands naturally as learners acquire knowledge. Pedagogical safety constraints are enforced by construction, with formal guarantees of structural prerequisite safety, primal-dual convergence, and strict dominance over post-hoc filtering. MC-CPO is the only method to reduce reward hacking severity across all conditions. Mean per-episode mastery gain increases by 18.3% on Junyi Academy and 54.0% on XES3G5M relative to all baselines, while competitive engagement performance is maintained. These results support structural constraint modelling as a principled foundation for safer adaptive instructional policies in deployed tutoring systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 39 references that are in the map. First posted on arXiv 5 Apr 2026.

Builds on

23 of 39 references.

References · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2016Concrete Problems in AI Safetycited by 1,582
  4. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  5. 2017Constrained Policy Optimizationcited by 928
  6. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  7. 2018Reward Constrained Policy Optimizationcited by 397
  8. 2015Deep Knowledge Tracingcited by 328
  9. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  10. 2022The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Modelscited by 312
  11. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
  12. 2017Inverse Reward Designcited by 246
11 more
  1. 2018Safe Exploration in Continuous Action Spacescited by 237
  2. 2022Defining and Characterizing Reward Hackingcited by 184
  3. 2020Graph Constrained Reinforcement Learning for Natural Language Action Spacescited by 80
  4. 2023State-wise Safe Reinforcement Learning: A Surveycited by 48
  5. 2024A Survey of Constraint Formulations in Safe Reinforcement Learningcited by 42
  6. 2021DOPE: Doubly Optimistic and Pessimistic Exploration for Safe Reinforcement Learningcited by 34
  7. 2023Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithmscited by 27
  8. 2025Detecting Proxy Gaming in RL and LLM Alignment via Evaluator Stress Testscited by 19
  9. 2023Constraint-Conditioned Policy Optimization for Versatile Safe Reinforcement Learningcited by 17
  10. 2023Towards Goal-oriented Intelligent Tutoring Systems in Online Educationcited by 7
  11. 2023Feasible Policy Iteration for Safe Reinforcement Learningcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Pedagogical Safety in Educational Reinforcement Learning: Formalizing and Detecting Reward Hacking in AI Tutoring Systems
  2. 2026PEARL: Training Socratic Tutors with Pedagogically Aligned Reinforcement Learning
  3. 2026Distill Skills into Weights, Not Prompts: Abstract Skills as Privileged Signals for On-Policy Self-Distillation
  4. 2026Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients
  5. 2026On-Policy Distillation Meets Off-Policy GRPO: Training Compact Instruction-Following Rerankers
  6. 2026A Good Self-Teacher Meets the Student Where They Are: Joint On-Policy Learning and Teaching
  7. 2026SocraticPO: Policy Optimization via Interactive Guidance
  8. 2026DiffGate: Difficulty-Gated Teacher Guidance for On-Policy Distillation