# MIRROR: Differentiable Deep Social Projection for Assistive Human-Robot Communication

arXiv: https://arxiv.org/abs/2203.02877 · HTML: https://mlsanity.com/paper/2203.02877/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 27 of its 53 references that are in the map.
- First posted on arXiv 6 Mar 2022.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 27 of 27)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Reinforcement Learning with Augmented Data](https://mlsanity.com/paper/2004.14990/) (2020)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Vision-and-Dialog Navigation](https://mlsanity.com/paper/1907.04957/) (2019)
- [Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model](https://mlsanity.com/paper/1907.00953/) (2019)
- [Multimodal Generative Models for Scalable Weakly-Supervised Learning](https://mlsanity.com/paper/1802.05335/) (2018)
- [Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks](https://mlsanity.com/paper/1810.10191/) (2018)
- [Modeling Missing Data in Clinical Time Series with RNNs](https://mlsanity.com/paper/1606.04130/) (2016)
- [Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior](https://mlsanity.com/paper/1805.08010/) (2018)
- [When Humans Aren't Optimal: Robots that Collaborate with Risk-Aware Humans](https://mlsanity.com/paper/2001.04377/) (2020)
- [Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recovery](https://mlsanity.com/paper/2101.01625/) (2021)
- [Residual Policy Learning for Shared Autonomy](https://mlsanity.com/paper/2004.05097/) (2020)
- [Cognitive science as a source of forward and inverse models of human decisions for robotics and control](https://mlsanity.com/paper/2109.00127/) (2021)
- [On the Utility of Model Learning in HRI](https://mlsanity.com/paper/1901.01291/) (2019)
- [A Review of Verbal and Non-Verbal Human-Robot Interactive Communication](https://mlsanity.com/paper/1401.4994/) (2014)
- [Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization](https://mlsanity.com/paper/2011.08541/) (2020)
- [Personalized Dynamics Models for Adaptive Assistive Navigation Systems](https://mlsanity.com/paper/1804.04118/) (2018)
- [Multi-Modal Mutual Information (MuMMI) Training for Robust Self-Supervised Deep Reinforcement Learning](https://mlsanity.com/paper/2107.02339/) (2021)
- [Multi-Task Trust Transfer for Human-Robot Interaction](https://mlsanity.com/paper/1807.01866/) (2018)
- [Factorized Inference in Deep Markov Models for Incomplete Multimodal Time Series](https://mlsanity.com/paper/1905.13570/) (2019)
- [Assisted Perception: Optimizing Observations to Communicate State](https://mlsanity.com/paper/2008.02840/) (2020)
- [Getting to Know One Another: Calibrating Intent, Capabilities and Trust for Human-Robot Collaboration](https://mlsanity.com/paper/2008.00699/) (2020)
- [Robot Capability and Intention in Trust-based Decisions across Tasks](https://mlsanity.com/paper/1909.05329/) (2019)

## Built on by (most cited first, 7 of 7)

- [Translating Natural Language to Planning Goals with Large-Language Models](https://mlsanity.com/paper/2302.05128/) (2023)
- [Large Language Models as Zero-Shot Human Models for Human-Robot Interaction](https://mlsanity.com/paper/2303.03548/) (2023)
- [Graph-enabled Reinforcement Learning for Time Series Forecasting with Adaptive Intelligence](https://mlsanity.com/paper/2309.10186/) (2023)
- [Shared Autonomy for Proximal Teaching](https://mlsanity.com/paper/2502.19899/) (2025)
- [Modeling Mobile Health Users as Reinforcement Learning Agents](https://mlsanity.com/paper/2212.00863/) (2022)
- [Latent Emission-Augmented Perspective-Taking (LEAPT) for Human-Robot Interaction](https://mlsanity.com/paper/2308.06498/) (2023)
- [Performance Asymmetry in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2505.19698/) (2025)

## Abstract (arXiv)

Communication is a hallmark of intelligence. In this work, we present MIRROR, an approach to (i) quickly learn human models from human demonstrations, and (ii) use the models for subsequent communication planning in assistive shared-control settings. MIRROR is inspired by social projection theory, which hypothesizes that humans use self-models to understand others. Likewise, MIRROR leverages self-models learned using reinforcement learning to bootstrap human modeling. Experiments with simulated humans show that this approach leads to rapid learning and more robust models compared to existing behavioral cloning and state-of-the-art imitation learning methods. We also present a human-subject study using the CARLA simulator which shows that (i) MIRROR is able to scale to complex domains with high-dimensional observations and complicated world physics and (ii) provides effective assistive communication that enabled participants to drive more safely in adverse weather conditions.