mlsanityData to 14 Oct 2026

Optimal Behavior Prior: Data-Efficient Human Models for Improved Human-AI Collaboration

arXiv 2211.01602 ↗First posted 3 Nov 2022cs.LG cs.HCcited by 8

Not on the map. Why?

Abstract (arXiv)

AI agents designed to collaborate with people benefit from models that enable them to anticipate human behavior. However, realistic models tend to require vast amounts of human data, which is often hard to collect. A good prior or initialization could make for more data-efficient training, but what makes for a good prior on human behavior? Our work leverages a very simple assumption: people generally act closer to optimal than to random chance. We show that using optimal behavior as a prior for human models makes these models vastly more data-efficient and able to generalize to new environments. Our intuition is that such a prior enables the training to focus one's precious real-world data on capturing the subtle nuances of human suboptimality, instead of on the basics of how to do the task in the first place. We also show that using these improved human models often leads to better human-AI collaboration performance compared to using models based on real human data alone.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 26 of its 49 references that are in the map. First posted on arXiv 3 Nov 2022.

Builds on

26 of 49 references.

References · 26

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  3. 2018Behavioral Cloning from Observationcited by 492
  4. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  5. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
  6. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
  7. 2014CNN Features off-the-shelf: an Astounding Baseline for Recognitioncited by 316
  8. 2020"Other-Play" for Zero-Shot Coordinationcited by 198
  9. 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
  10. 2021Collaborating with Humans without Human Datacited by 142
  11. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 141
  12. 2020Watch-And-Help: A Challenge for Social Perception and Human-AI Collaborationcited by 133
14 more
  1. 2020Aligning Superhuman AI with Human Behavior: Chess as a Model Systemcited by 79
  2. 2021Modeling Strong and Human-Like Gameplay with KL-Regularized Searchcited by 50
  3. 2019Simplified Action Decoder for Deep Multi-Agent Reinforcement Learningcited by 48
  4. 2015A Game-Theoretic Model and Best-Response Learning Method for Ad Hoc Coordination in Multiagent Systemscited by 44
  5. 2021Creating Multimodal Interactive Agents with Imitation and Self-Supervised Learningcited by 40
  6. 2021On the Importance of Environments in Human-Robot Coordinationcited by 37
  7. 2020Too many cooks: Bayesian inference for coordinating multi-agent collaborationcited by 29
  8. 2021Evaluating the Robustness of Collaborative Agentscited by 25
  9. 2015Early Stopping is Nonparametric Variational Inferencecited by 23
  10. 2019On the Utility of Model Learning in HRIcited by 21
  11. 2022CausalAgents: A Robustness Benchmark for Motion Forecasting using Causal Relationshipscited by 18
  12. 2020Adversarially Guided Self-Play for Adopting Social Conventionscited by 10
  13. 2022Human-AI Coordination via Human-Regularized Search and Learningcited by 8
  14. 2022Assisting Unknown Teammates in Unknown Tasks: Ad Hoc Teamwork under Partial Observabilitycited by 6

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3; 2025: 4.

8 papers.

  1. 2024Evaluating Human-AI Collaboration: A Review and Methodological Frameworkcited by 27
  2. 2025OvercookedV2: Rethinking Overcooked for Zero-Shot Coordinationcited by 16
  3. 2024The Overcooked Generalisation Challenge: Evaluating Cooperation with Novel Partners in Unknown Environments Using Unsupervised Environment Designcited by 11
  4. 2024Designing Skill-Compatible AI: Methodologies and Frameworks in Chesscited by 10
  5. 2025AssistanceZero: Scalably Solving Assistance Gamescited by 10
  6. 2023A behavioural transformer for effective collaboration between a robot and a non-stationary humancited by 2
  7. 2025Automatic Curriculum Design for Zero-Shot Human-AI Coordinationcited by 1
  8. 2025Towards Robust Deep Reinforcement Learning against Environmental State Perturbationcited by 0