# AI Coaching for Accelerating Human Skill Development with Reinforcement Learning

arXiv: https://arxiv.org/abs/2606.25337 · HTML: https://mlsanity.com/paper/2606.25337/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 24 Jun 2026.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [The Complexity of Decentralized Control of Markov Decision Processes](https://mlsanity.com/paper/1301.3836/) (2013)
- [Shared Autonomy via Deep Reinforcement Learning](https://mlsanity.com/paper/1802.01744/) (2018)
- [Modeling Strong and Human-Like Gameplay with KL-Regularized Search](https://mlsanity.com/paper/2112.07544/) (2021)
- [Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development](https://mlsanity.com/paper/2501.16946/) (2025)
- [Pragmatic-Pedagogic Value Alignment](https://mlsanity.com/paper/1707.06354/) (2017)
- [Stochastic Dynamic Games in Belief Space](https://mlsanity.com/paper/1909.06963/) (2019)
- [Learning to Play Trajectory Games Against Opponents with Unknown Objectives](https://mlsanity.com/paper/2211.13779/) (2022)
- [Deception Game: Closing the Safety-Learning Loop in Interactive Robot Autonomy](https://mlsanity.com/paper/2309.01267/) (2023)
- [AssistanceZero: Scalably Solving Assistance Games](https://mlsanity.com/paper/2504.07091/) (2025)
- [Safety with Agency: Human-Centered Safety Filter with Application to AI-Assisted Motorsports](https://mlsanity.com/paper/2504.11717/) (2025)
- [Dreaming to Assist: Learning to Align with Human Objectives for Shared Control in High-Speed Racing](https://mlsanity.com/paper/2410.10062/) (2024)
- [Computational Teaching for Driving via Multi-Task Imitation Learning](https://mlsanity.com/paper/2410.01608/) (2024)
- [Shared Autonomy for Proximal Teaching](https://mlsanity.com/paper/2502.19899/) (2025)
- [Agile Flight Emerges from Multi-Agent Competitive Racing](https://mlsanity.com/paper/2512.11781/) (2025)
- [Efficient and Reliable Teleoperation through Real-to-Sim-to-Real Shared Autonomy](https://mlsanity.com/paper/2603.17016/) (2026)
- [Reinforcement Learning for Shared Autonomy Drone Landings](https://mlsanity.com/paper/2202.02927/) (2022)
- [Cyber Racing Coach: A Haptic Shared Control Framework for Teaching Advanced Driving Skills](https://mlsanity.com/paper/2509.20653/) (2025)

## Abstract (arXiv)

AI copilots can substantially boost human performance through shared control, but excessive assistance can induce over-reliance and skill atrophy. This paper studies how an embodied AI agent can act as a coach that accelerates human motor-skill development. We argue that effective coaching requires strategic scaffolding and stepping back that are aligned with the learner's capability, allowing productive failures that drive learning. We formalize the interactive AI coaching process as a non-cooperative dynamic game in which the learner optimizes task performance while the coach targets the learner's independent competence. Building on this formalism, we develop a reinforcement learning framework combining adaptive shared control with probabilistic models of the coach's causal influence on skill evolution, enabling tractable training of coaching policies. A comprehensive user study (N=33) on first-person-view drone racing shows significant gains in human learning outcomes over state-of-the-art AI coaching baselines.