# Towards Modeling and Influencing the Dynamics of Human Learning

arXiv: https://arxiv.org/abs/2301.00901 · HTML: https://mlsanity.com/paper/2301.00901/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 58 references that are in the map.
- First posted on arXiv 2 Jan 2023.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Offline Reinforcement Learning as One Big Sequence Modeling Problem](https://mlsanity.com/paper/2106.02039/) (2021)
- [Learning Latent Representations to Influence Multi-Agent Interaction](https://mlsanity.com/paper/2011.06619/) (2020)
- [Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior](https://mlsanity.com/paper/1805.08010/) (2018)
- [Enabling Robots to Communicate their Objectives](https://mlsanity.com/paper/1702.03465/) (2017)
- [Trust-Aware Decision Making for Human-Robot Collaboration: Model Learning and Planning](https://mlsanity.com/paper/1801.04099/) (2018)
- [Game-Theoretic Modeling of Human Adaptation in Human-Robot Collaboration](https://mlsanity.com/paper/1701.07790/) (2017)
- [Safety Assurances for Human-Robot Interaction via Confidence-aware Game-theoretic Human Models](https://mlsanity.com/paper/2109.14700/) (2021)
- [ROIAL: Region of Interest Active Learning for Characterizing Exoskeleton Gait Preference Landscapes](https://mlsanity.com/paper/2011.04812/) (2020)
- [The Assistive Multi-Armed Bandit](https://mlsanity.com/paper/1901.08654/) (2019)
- [Infinite-Horizon Differentiable Model Predictive Control](https://mlsanity.com/paper/2001.02244/) (2020)
- [Learning Latent Actions to Control Assistive Robots](https://mlsanity.com/paper/2107.02907/) (2021)
- [Inferring Objectives in Continuous Dynamic Games from Noise-Corrupted Partial State Observations](https://mlsanity.com/paper/2106.03611/) (2021)
- [HARMONIC: A Multimodal Dataset of Assistive Human-Robot Collaboration](https://mlsanity.com/paper/1807.11154/) (2018)
- [Assisted Perception: Optimizing Observations to Communicate State](https://mlsanity.com/paper/2008.02840/) (2020)
- [Active Uncertainty Reduction for Human-Robot Interaction: An Implicit Dual Control Approach](https://mlsanity.com/paper/2202.07720/) (2022)
- [Assistive Teaching of Motor Control Tasks to Humans](https://mlsanity.com/paper/2211.14003/) (2022)
- [Multi-Hypothesis Interactions in Game-Theoretic Motion Planning](https://mlsanity.com/paper/2011.06047/) (2020)
- [Encouraging Human Interaction with Robot Teams: Legible and Fair Subtask Allocations](https://mlsanity.com/paper/2205.03477/) (2022)

## Built on by (most cited first, 11 of 11)

- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [Bootstrapping Cognitive Agents with a Large Language Model](https://mlsanity.com/paper/2403.00810/) (2024)
- [AI-Assisted Decision Making with Human Learning](https://mlsanity.com/paper/2502.13062/) (2025)
- [When Are Two Lists Better than One?: Benefits and Harms in Joint Decision-making](https://mlsanity.com/paper/2308.11721/) (2023)
- [Multi-Round Human-AI Collaboration with User-Specified Requirements](https://mlsanity.com/paper/2602.17646/) (2026)
- [CANINE: Coaching Visually Impaired Users for Interactive Navigation with a Robot Guide Dog](https://mlsanity.com/paper/2605.19501/) (2026)
- [Strategic Decision Support for AI Agents](https://mlsanity.com/paper/2606.12587/) (2026)
- [Robots Influencing Humans to Reveal their Goals during Collaboration and Competition](https://mlsanity.com/paper/2609.05519/) (2026)
- [PACE: A Framework for Learning and Control in Linear Incomplete-Information Differential Games](https://mlsanity.com/paper/2504.17128/) (2025)
- [Peer-Aware Cost Estimation in Nonlinear General-Sum Dynamic Games for Mutual Learning and Intent Inference](https://mlsanity.com/paper/2504.17129/) (2025)
- [Proximal State Nudging: Reducing Skill Atrophy from AI Assistance](https://mlsanity.com/paper/2605.20355/) (2026)

## Abstract (arXiv)

Humans have internal models of robots (like their physical capabilities), the world (like what will happen next), and their tasks (like a preferred goal). However, human internal models are not always perfect: for example, it is easy to underestimate a robot's inertia. Nevertheless, these models change and improve over time as humans gather more experience. Interestingly, robot actions influence what this experience is, and therefore influence how people's internal models change. In this work we take a step towards enabling robots to understand the influence they have, leverage it to better assist people, and help human models more quickly align with reality. Our key idea is to model the human's learning as a nonlinear dynamical system which evolves the human's internal model given new observations. We formulate a novel optimization problem to infer the human's learning dynamics from demonstrations that naturally exhibit human learning. We then formalize how robots can influence human learning by embedding the human's learning dynamics model into the robot planning problem. Although our formulations provide concrete problem statements, they are intractable to solve in full generality. We contribute an approximation that sacrifices the complexity of the human internal models we can represent, but enables robots to learn the nonlinear dynamics of these internal models. We evaluate our inference and planning methods in a suite of simulated environments and an in-person user study, where a 7DOF robotic arm teaches participants to be better teleoperators. While influencing human learning remains an open problem, our results demonstrate that this influence is possible and can be helpful in real human-robot interaction.