# Residual Policy Learning for Shared Autonomy

arXiv: https://arxiv.org/abs/2004.05097 · HTML: https://mlsanity.com/paper/2004.05097/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2020.

## Built on by (most cited first, 26 of 26)

- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [LILA: Language-Informed Latent Actions](https://mlsanity.com/paper/2111.03205/) (2021)
- [To the Noise and Back: Diffusion for Shared Autonomy](https://mlsanity.com/paper/2302.12244/) (2023)
- [Don't Start from Scratch: Behavioral Refinement via Interpolant-based Policy Diffusion](https://mlsanity.com/paper/2402.16075/) (2024)
- [Composable Energy Policies for Reactive Motion Generation and Reinforcement Learning](https://mlsanity.com/paper/2105.04962/) (2021)
- [Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition](https://mlsanity.com/paper/2407.00299/) (2024)
- [ASHA: Assistive Teleoperation via Human-in-the-Loop Reinforcement Learning](https://mlsanity.com/paper/2202.02465/) (2022)
- [First Contact: Unsupervised Human-Machine Co-Adaptation via Mutual Information Maximization](https://mlsanity.com/paper/2205.12381/) (2022)
- [MIRROR: Differentiable Deep Social Projection for Assistive Human-Robot Communication](https://mlsanity.com/paper/2203.02877/) (2022)
- [Retrieve, Don't Retrain: Extending Vision Language Action Models to New Tasks at Test Time](https://mlsanity.com/paper/2606.15631/) (2026)
- [X2T: Training an X-to-Text Typing Interface with Online Learning from User Feedback](https://mlsanity.com/paper/2203.02072/) (2022)
- [Human-AI Shared Control via Policy Dissection](https://mlsanity.com/paper/2206.00152/) (2022)
- [PeRP: Personalized Residual Policies For Congestion Mitigation Through Co-operative Advisory Systems](https://mlsanity.com/paper/2308.00864/) (2023)
- [On Optimizing Interventions in Shared Autonomy](https://mlsanity.com/paper/2112.09169/) (2021)
- [Residual Policy Learning for Powertrain Control](https://mlsanity.com/paper/2212.07611/) (2022)
- [Shared Autonomy with IDA: Interventional Diffusion Assistance](https://mlsanity.com/paper/2409.15317/) (2024)
- [FlashBack: Consistency Model-Accelerated Shared Autonomy](https://mlsanity.com/paper/2505.16892/) (2025)
- [Casper: Inferring Diverse Intents for Assistive Teleoperation with Vision Language Models](https://mlsanity.com/paper/2506.14727/) (2025)
- [Learning Latent Traits for Simulated Cooperative Driving Tasks](https://mlsanity.com/paper/2207.09619/) (2022)
- [Bootstrapping Adaptive Human-Machine Interfaces with Offline Reinforcement Learning](https://mlsanity.com/paper/2309.03839/) (2023)
- [Shared Control of Holonomic Wheelchairs through Reinforcement Learning](https://mlsanity.com/paper/2507.17055/) (2025)
- [End-to-end Optimization of Belief and Policy Learning in Shared Autonomy Paradigms](https://mlsanity.com/paper/2601.23285/) (2026)
- [AI-IoT-Robotics Integration: Survey of Frameworks, Emerging Trends, and the Path Toward Connected Robotics](https://mlsanity.com/paper/2606.01015/) (2026)
- [DexCompose: Reusing Dexterous Policies for Multi-Task Manipulation with a Single Hand](https://mlsanity.com/paper/2606.28323/) (2026)
- [Augmenting Human Performance with an XR Agent Learning from Online Behavior and BCI Evidence](https://mlsanity.com/paper/2608.30369/) (2026)
- [DexFLEX: Contact-Aware Foundation Controller for Command-Guided Dexterity](https://mlsanity.com/paper/2610.14372/) (2026)

## Abstract (arXiv)

Shared autonomy provides an effective framework for human-robot collaboration that takes advantage of the complementary strengths of humans and robots to achieve common goals. Many existing approaches to shared autonomy make restrictive assumptions that the goal space, environment dynamics, or human policy are known a priori, or are limited to discrete action spaces, preventing those methods from scaling to complicated real world environments. We propose a model-free, residual policy learning algorithm for shared autonomy that alleviates the need for these assumptions. Our agents are trained to minimally adjust the human's actions such that a set of goal-agnostic constraints are satisfied. We test our method in two continuous control environments: Lunar Lander, a 2D flight control domain, and a 6-DOF quadrotor reaching task. In experiments with human and surrogate pilots, our method significantly improves task performance without any knowledge of the human's goal beyond the constraints. These results highlight the ability of model-free deep reinforcement learning to realize assistive agents suited to continuous control settings with little knowledge of user intent.