# A behavioural transformer for effective collaboration between a robot and a non-stationary human

arXiv: https://arxiv.org/abs/2307.13447 · HTML: https://mlsanity.com/paper/2307.13447/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 30 references that are in the map.
- First posted on arXiv 25 Jul 2023.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks](https://mlsanity.com/paper/2006.07869/) (2020)
- [A Comparative Study on Transformer vs RNN in Speech Applications](https://mlsanity.com/paper/1909.06317/) (2019)
- [Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problems](https://mlsanity.com/paper/1709.08071/) (2017)
- [Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1906.04737/) (2019)
- [Learning Latent Representations to Influence Multi-Agent Interaction](https://mlsanity.com/paper/2011.06619/) (2020)
- [The continuous Bernoulli: fixing a pervasive error in variational autoencoders](https://mlsanity.com/paper/1907.06845/) (2019)
- [Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior](https://mlsanity.com/paper/1805.08010/) (2018)
- [The Boltzmann Policy Distribution: Accounting for Systematic Suboptimality in Human Models](https://mlsanity.com/paper/2204.10759/) (2022)
- [Human irrationality: both bad and good for reward inference](https://mlsanity.com/paper/2111.06956/) (2021)
- [Influencing Towards Stable Multi-Agent Interactions](https://mlsanity.com/paper/2110.08229/) (2021)
- [Influencing Long-Term Behavior in Multiagent Reinforcement Learning](https://mlsanity.com/paper/2203.03535/) (2022)
- [LESS is More: Rethinking Probabilistic Models of Human Behavior](https://mlsanity.com/paper/2001.04465/) (2020)
- [Human-Robot Mutual Adaptation in Shared Autonomy](https://mlsanity.com/paper/1701.07851/) (2017)
- [RILI: Robustly Influencing Latent Intent](https://mlsanity.com/paper/2203.12705/) (2022)
- [Optimal Behavior Prior: Data-Efficient Human Models for Improved Human-AI Collaboration](https://mlsanity.com/paper/2211.01602/) (2022)
- [Multimodal Meta-Learning for Time Series Regression](https://mlsanity.com/paper/2108.02842/) (2021)

## Built on by (most cited first, 2 of 2)

- [Partner Modelling Emerges in Recurrent Agents (But Only When It Matters)](https://mlsanity.com/paper/2505.17323/) (2025)
- [Enabling robots to follow abstract instructions and complete complex dynamic tasks](https://mlsanity.com/paper/2406.11231/) (2024)

## Abstract (arXiv)

A key challenge in human-robot collaboration is the non-stationarity created by humans due to changes in their behaviour. This alters environmental transitions and hinders human-robot collaboration. We propose a principled meta-learning framework to explore how robots could better predict human behaviour, and thereby deal with issues of non-stationarity. On the basis of this framework, we developed Behaviour-Transform (BeTrans). BeTrans is a conditional transformer that enables a robot agent to adapt quickly to new human agents with non-stationary behaviours, due to its notable performance with sequential data. We trained BeTrans on simulated human agents with different systematic biases in collaborative settings. We used an original customisable environment to show that BeTrans effectively collaborates with simulated human agents and adapts faster to non-stationary simulated human agents than SOTA techniques.