# First Contact: Unsupervised Human-Machine Co-Adaptation via Mutual Information Maximization

arXiv: https://arxiv.org/abs/2205.12381 · HTML: https://mlsanity.com/paper/2205.12381/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 79 references that are in the map.
- First posted on arXiv 24 May 2022.

## Builds on (most cited first, 30 of 32)

- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [Dynamics-Aware Unsupervised Discovery of Skills](https://mlsanity.com/paper/1907.01657/) (2019)
- [Randomized Ensembled Double Q-Learning: Learning Fast Without a Model](https://mlsanity.com/paper/2101.05982/) (2021)
- [Variational Intrinsic Control](https://mlsanity.com/paper/1611.07507/) (2016)
- [Contextual Markov Decision Processes](https://mlsanity.com/paper/1502.02259/) (2015)
- [MediaPipe Hands: On-device Real-time Hand Tracking](https://mlsanity.com/paper/2006.10214/) (2020)
- [Interactive Learning from Policy-Dependent Human Feedback](https://mlsanity.com/paper/1701.06049/) (2017)
- [Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning](https://mlsanity.com/paper/1509.08731/) (2015)
- [Fast Task Inference with Variational Intrinsic Successor Features](https://mlsanity.com/paper/1906.05030/) (2019)
- [Unsupervised Control Through Non-Parametric Discriminative Rewards](https://mlsanity.com/paper/1811.11359/) (2018)
- [Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control](https://mlsanity.com/paper/1611.02796/) (2016)
- [Putting An End to End-to-End: Gradient-Isolated Learning of Representations](https://mlsanity.com/paper/1905.11786/) (2019)
- [CIC: Contrastive Intrinsic Control for Unsupervised Skill Discovery](https://mlsanity.com/paper/2202.00161/) (2022)
- [Shared Autonomy via Deep Reinforcement Learning](https://mlsanity.com/paper/1802.01744/) (2018)
- [Biases for Emergent Communication in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/1912.05676/) (2019)
- [Learning Language Games through Interaction](https://mlsanity.com/paper/1606.02447/) (2016)
- [Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior](https://mlsanity.com/paper/1805.08010/) (2018)
- [The EMPATHIC Framework for Task Learning from Implicit Human Feedback](https://mlsanity.com/paper/2009.13649/) (2020)
- [Shared Autonomy with Learned Latent Actions](https://mlsanity.com/paper/2005.03210/) (2020)
- [Policy Evaluation Networks](https://mlsanity.com/paper/2002.11833/) (2020)
- [AvE: Assistance via Empowerment](https://mlsanity.com/paper/2006.14796/) (2020)
- [Residual Policy Learning for Shared Autonomy](https://mlsanity.com/paper/2004.05097/) (2020)
- [Learning Adaptive Language Interfaces through Decomposition](https://mlsanity.com/paper/2010.05190/) (2020)
- [Translating Neuralese](https://mlsanity.com/paper/1704.06960/) (2017)
- [Learning Models for Shared Control of Human-Machine Systems with Unknown Dynamics](https://mlsanity.com/paper/1808.08268/) (2018)
- [ASHA: Assistive Teleoperation via Human-in-the-Loop Reinforcement Learning](https://mlsanity.com/paper/2202.02465/) (2022)
- [Autonomy Infused Teleoperation with Application to BCI Manipulation](https://mlsanity.com/paper/1503.05451/) (2015)
- [Data-Efficient Mutual Information Neural Estimator](https://mlsanity.com/paper/1905.03319/) (2019)
- [Neuroprosthetic decoder training as imitation learning](https://mlsanity.com/paper/1511.04156/) (2015)
- [Accelerating Reinforcement Learning Agent with EEG-based Implicit Human Feedback](https://mlsanity.com/paper/2006.16498/) (2020)

## Built on by (most cited first, 8 of 8)

- [To the Noise and Back: Diffusion for Shared Autonomy](https://mlsanity.com/paper/2302.12244/) (2023)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [Enhancing Intent Understanding for Ambiguous prompt: A Human-Machine Co-Adaption Strategy](https://mlsanity.com/paper/2501.15167/) (2025)
- [Learning to Assist Humans without Inferring Rewards](https://mlsanity.com/paper/2411.02623/) (2024)
- [PVPUFormer: Probabilistic Visual Prompt Unified Transformer for Interactive Image Segmentation](https://mlsanity.com/paper/2306.06656/) (2023)
- [Self-Calibrating BCIs: Ranking and Recovery of Mental Targets Without Labels](https://mlsanity.com/paper/2506.11151/) (2025)
- [Scene Graph-guided SegCaptioning Transformer with Fine-grained Alignment for Controllable Video Segmentation and Captioning](https://mlsanity.com/paper/2603.20887/) (2026)
- [A Fair Objective for Human-Empowerment-Preserving AI: Desiderata, Design, and Likely Behavioral Consequences](https://mlsanity.com/paper/2608.08240/) (2026)

## Abstract (arXiv)

How can we train an assistive human-machine interface (e.g., an electromyography-based limb prosthesis) to translate a user's raw command signals into the actions of a robot or computer when there is no prior mapping, we cannot ask the user for supervision in the form of action labels or reward feedback, and we do not have prior knowledge of the tasks the user is trying to accomplish? The key idea in this paper is that, regardless of the task, when an interface is more intuitive, the user's commands are less noisy. We formalize this idea as a completely unsupervised objective for optimizing interfaces: the mutual information between the user's command signals and the induced state transitions in the environment. To evaluate whether this mutual information score can distinguish between effective and ineffective interfaces, we conduct an observational study on 540K examples of users operating various keyboard and eye gaze interfaces for typing, controlling simulated robots, and playing video games. The results show that our mutual information scores are predictive of the ground-truth task completion metrics in a variety of domains, with an average Spearman's rank correlation of 0.43. In addition to offline evaluation of existing interfaces, we use our unsupervised objective to learn an interface from scratch: we randomly initialize the interface, have the user attempt to perform their desired tasks using the interface, measure the mutual information score, and update the interface to maximize mutual information through reinforcement learning. We evaluate our method through a user study with 12 participants who perform a 2D cursor control task using a perturbed mouse, and an experiment with one user playing the Lunar Lander game using hand gestures. The results show that we can learn an interface from scratch, without any user supervision or prior knowledge of tasks, in under 30 minutes.