# Learning Representations that Enable Generalization in Assistive Tasks

arXiv: https://arxiv.org/abs/2212.03175 · HTML: https://mlsanity.com/paper/2212.03175/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 5 Dec 2022.

## Builds on (most cited first, 25 of 25)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images](https://mlsanity.com/paper/1506.07365/) (2015)
- [Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics](https://mlsanity.com/paper/1703.09312/) (2017)
- [Human-level performance in first-person multiplayer games with population-based deep reinforcement learning](https://mlsanity.com/paper/1807.01281/) (2018)
- [MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale](https://mlsanity.com/paper/2104.08212/) (2021)
- [Collaborating with Humans without Human Data](https://mlsanity.com/paper/2110.08176/) (2021)
- [Open-ended Learning in Symmetric Zero-sum Games](https://mlsanity.com/paper/1901.08106/) (2019)
- [Learning Latent Representations to Influence Multi-Agent Interaction](https://mlsanity.com/paper/2011.06619/) (2020)
- [Learning to Manipulate Deformable Objects without Demonstrations](https://mlsanity.com/paper/1910.13439/) (2019)
- [Unpacking Reward Shaping: Understanding the Benefits of Reward Engineering on Sample Complexity](https://mlsanity.com/paper/2210.09579/) (2022)
- [Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrifices](https://mlsanity.com/paper/2008.02790/) (2020)
- [Sim-to-Real Transfer for Biped Locomotion](https://mlsanity.com/paper/1903.01390/) (2019)
- [Assistive Gym: A Physics Simulation Framework for Assistive Robotics](https://mlsanity.com/paper/1910.04700/) (2019)
- [Deep Imitation Learning of Sequential Fabric Smoothing From an Algorithmic Supervisor](https://mlsanity.com/paper/1910.04854/) (2019)
- [Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learning](https://mlsanity.com/paper/2107.03961/) (2021)
- [Learning to Collaborate from Simulation for Robot-Assisted Dressing](https://mlsanity.com/paper/1909.06682/) (2019)
- [Invariance Through Latent Alignment](https://mlsanity.com/paper/2112.08526/) (2021)
- [RILI: Robustly Influencing Latent Intent](https://mlsanity.com/paper/2203.12705/) (2022)
- [Assistive VR Gym: Interactions with Real People to Improve Virtual Assistive Robots](https://mlsanity.com/paper/2007.04959/) (2020)

## Built on by (most cited first, 13 of 13)

- [Learning Vision-based Pursuit-Evasion Robot Policies](https://mlsanity.com/paper/2308.16185/) (2023)
- [FEAST: A Flexible Mealtime-Assistance System Towards In-the-Wild Personalization](https://mlsanity.com/paper/2506.14968/) (2025)
- [Select to Perfect: Imitating desired behavior from large multi-agent data](https://mlsanity.com/paper/2405.03735/) (2024)
- [GRACE: Generalizing Robot-Assisted Caregiving with User Functionality Embeddings](https://mlsanity.com/paper/2501.17855/) (2025)
- [Bootstrapping Linear Models for Fast Online Adaptation in Human-Agent Collaboration](https://mlsanity.com/paper/2404.10733/) (2024)
- [Aligning Generalisation Between Humans and Machines](https://mlsanity.com/paper/2411.15626/) (2024)
- [Quantifying Assistive Robustness Via the Natural-Adversarial Frontier](https://mlsanity.com/paper/2310.10610/) (2023)
- [Robotic Learning in your Backyard: A Neural Simulator from Open Source Components](https://mlsanity.com/paper/2410.19564/) (2024)
- [Coloring Between the Lines: Personalization in the Null Space of Planning Constraints](https://mlsanity.com/paper/2505.15503/) (2025)
- [Robustifying a Policy in Multi-Agent RL with Diverse Cooperative Behaviors and Adversarial Style Sampling for Assistive Tasks](https://mlsanity.com/paper/2403.00344/) (2024)
- [Reducing Risk for Assistive Reinforcement Learning Policies with Diffusion Models](https://mlsanity.com/paper/2405.07603/) (2024)
- [Using High-Level Patterns to Estimate How Humans Predict a Robot will Behave](https://mlsanity.com/paper/2409.13533/) (2024)
- [Dynamics Are Learned, Not Told: Semi-Supervised Discovery of Latent Dynamics Geometries For Zero-Shot Policy Adaptation](https://mlsanity.com/paper/2606.02280/) (2026)

## Abstract (arXiv)

Recent work in sim2real has successfully enabled robots to act in physical environments by training in simulation with a diverse ''population'' of environments (i.e. domain randomization). In this work, we focus on enabling generalization in assistive tasks: tasks in which the robot is acting to assist a user (e.g. helping someone with motor impairments with bathing or with scratching an itch). Such tasks are particularly interesting relative to prior sim2real successes because the environment now contains a human who is also acting. This complicates the problem because the diversity of human users (instead of merely physical environment parameters) is more difficult to capture in a population, thus increasing the likelihood of encountering out-of-distribution (OOD) human policies at test time. We advocate that generalization to such OOD policies benefits from (1) learning a good latent representation for human policies that test-time humans can accurately be mapped to, and (2) making that representation adaptable with test-time interaction data, instead of relying on it to perfectly capture the space of human policies based on the simulated population only. We study how to best learn such a representation by evaluating on purposefully constructed OOD test policies. We find that sim2real methods that encode environment (or population) parameters and work well in tasks that robots do in isolation, do not work well in assistance. In assistance, it seems crucial to train the representation based on the history of interaction directly, because that is what the robot will have access to at test time. Further, training these representations to then predict human actions not only gives them better structure, but also enables them to be fine-tuned at test-time, when the robot observes the partner act. https://adaptive-caregiver.github.io.