# Efficient Model Learning for Human-Robot Collaborative Tasks

arXiv: https://arxiv.org/abs/1405.6341 · HTML: https://mlsanity.com/paper/1405.6341/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 May 2014.

## Built on by (most cited first, 30 of 35)

- [Semantics for Robotic Mapping, Perception and Interaction: A Survey](https://mlsanity.com/paper/2101.00443/) (2021)
- [On the Importance of Environments in Human-Robot Coordination](https://mlsanity.com/paper/2106.10853/) (2021)
- [The Utility of Explainable AI in Ad Hoc Human-Machine Teaming](https://mlsanity.com/paper/2209.03943/) (2022)
- [Social Coordination and Altruism in Autonomous Driving](https://mlsanity.com/paper/2107.00200/) (2021)
- [NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistants](https://mlsanity.com/paper/2301.05223/) (2023)
- [Human-robot collaboration and machine learning: a systematic review of recent research](https://mlsanity.com/paper/2110.07448/) (2021)
- [Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration](https://mlsanity.com/paper/2108.06038/) (2021)
- [Reinforcement Learning Under Algorithmic Triage](https://mlsanity.com/paper/2109.11328/) (2021)
- [Fast Lifelong Adaptive Inverse Reinforcement Learning from Demonstrations](https://mlsanity.com/paper/2209.11908/) (2022)
- [Coordination with Humans via Strategy Matching](https://mlsanity.com/paper/2210.15099/) (2022)
- [Conditional Imitation Learning for Multi-Agent Games](https://mlsanity.com/paper/2201.01448/) (2022)
- [Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmas](https://mlsanity.com/paper/2402.17270/) (2024)
- [Altruistic Maneuver Planning for Cooperative Autonomous Vehicles Using Multi-agent Advantage Actor-Critic](https://mlsanity.com/paper/2107.05664/) (2021)
- [Subverting machines, fluctuating identities: Re-learning human categorization](https://mlsanity.com/paper/2205.13740/) (2022)
- [Not all users are the same: Providing personalized explanations for sequential decision making problems](https://mlsanity.com/paper/2106.12207/) (2021)
- [Prediction-aware and Reinforcement Learning based Altruistic Cooperative Driving](https://mlsanity.com/paper/2211.10585/) (2022)
- [Learning Latent Representations to Co-Adapt to Humans](https://mlsanity.com/paper/2212.09586/) (2022)
- [Ella: Embodied Social Agents with Lifelong Memory](https://mlsanity.com/paper/2506.24019/) (2025)
- [Adaptively Coordinating with Novel Partners via Learned Latent Strategies](https://mlsanity.com/paper/2511.12754/) (2025)
- [Inverse Reinforcement Learning Based Stochastic Driver Behavior Learning](https://mlsanity.com/paper/2107.06344/) (2021)
- [Contrastive Learning from Exploratory Actions: Leveraging Natural Interactions for Preference Elicitation](https://mlsanity.com/paper/2501.01367/) (2025)
- [Autonomous Human-Robot Interaction via Operator Imitation](https://mlsanity.com/paper/2504.02724/) (2025)
- [Inductive Learning of Robot Task Knowledge from Raw Data and Online Expert Feedback](https://mlsanity.com/paper/2501.07507/) (2025)
- [ProVox: Personalization and Proactive Planning for Situated Human-Robot Collaboration](https://mlsanity.com/paper/2506.12248/) (2025)
- [Strategy Discovery and Mixture in Lifelong Learning from Heterogeneous Demonstration](https://mlsanity.com/paper/2202.07014/) (2022)
- [Learning-based social coordination to improve safety and robustness of cooperative autonomous vehicles in mixed traffic](https://mlsanity.com/paper/2211.11963/) (2022)
- [Enabling robots to follow abstract instructions and complete complex dynamic tasks](https://mlsanity.com/paper/2406.11231/) (2024)
- [Improving User Experience in Preference-Based Optimization of Reward Functions for Assistive Robots](https://mlsanity.com/paper/2411.11182/) (2024)
- [Modeling Latent Partner Strategies for Adaptive Zero-Shot Human-Agent Collaboration](https://mlsanity.com/paper/2507.05244/) (2025)
- [Partner Capability Estimation for Task-Agnostic Adaptation in Ad-Hoc Teamwork](https://mlsanity.com/paper/2607.27177/) (2026)

## Abstract (arXiv)

We present a framework for learning human user models from joint-action demonstrations that enables the robot to compute a robust policy for a collaborative task with a human. The learning takes place completely automatically, without any human intervention. First, we describe the clustering of demonstrated action sequences into different human types using an unsupervised learning algorithm. These demonstrated sequences are also used by the robot to learn a reward function that is representative for each type, through the employment of an inverse reinforcement learning algorithm. The learned model is then used as part of a Mixed Observability Markov Decision Process formulation, wherein the human type is a partially observable variable. With this framework, we can infer, either offline or online, the human type of a new user that was not included in the training set, and can compute a policy for the robot that will be aligned to the preference of this new user and will be robust to deviations of the human actions from prior demonstrations. Finally we validate the approach using data collected in human subject experiments, and conduct proof-of-concept demonstrations in which a person performs a collaborative task with a small industrial robot.