# Shared Autonomy via Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1802.01744 · HTML: https://mlsanity.com/paper/1802.01744/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Feb 2018.

## Built on by (most cited first, 30 of 66)

- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning](https://mlsanity.com/paper/2109.08273/) (2021)
- [Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment](https://mlsanity.com/paper/2211.08416/) (2022)
- [TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction](https://mlsanity.com/paper/2405.10315/) (2024)
- [LazyDAgger: Reducing Context Switching in Interactive Imitation Learning](https://mlsanity.com/paper/2104.00053/) (2021)
- [Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimization](https://mlsanity.com/paper/2202.10341/) (2022)
- [Core Challenges in Embodied Vision-Language Planning](https://mlsanity.com/paper/2106.13948/) (2021)
- [LILA: Language-Informed Latent Actions](https://mlsanity.com/paper/2111.03205/) (2021)
- [The transformative potential of machine learning for experiments in fluid mechanics](https://mlsanity.com/paper/2303.15832/) (2023)
- [On the Sensitivity of Reward Inference to Misspecified Human Models](https://mlsanity.com/paper/2212.04717/) (2022)
- [To the Noise and Back: Diffusion for Shared Autonomy](https://mlsanity.com/paper/2302.12244/) (2023)
- [Learning from Active Human Involvement through Proxy Value Propagation](https://mlsanity.com/paper/2502.03369/) (2025)
- [Situational Confidence Assistance for Lifelong Shared Autonomy](https://mlsanity.com/paper/2104.06556/) (2021)
- [Reinforcement Learning Under Algorithmic Triage](https://mlsanity.com/paper/2109.11328/) (2021)
- [Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computation](https://mlsanity.com/paper/2203.07322/) (2022)
- [Learning to Assist Humans without Inferring Rewards](https://mlsanity.com/paper/2411.02623/) (2024)
- [Koopman-Assisted Reinforcement Learning](https://mlsanity.com/paper/2403.02290/) (2024)
- [Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition](https://mlsanity.com/paper/2407.00299/) (2024)
- [Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanism](https://mlsanity.com/paper/2506.09176/) (2025)
- [ASHA: Assistive Teleoperation via Human-in-the-Loop Reinforcement Learning](https://mlsanity.com/paper/2202.02465/) (2022)
- [First Contact: Unsupervised Human-Machine Co-Adaptation via Mutual Information Maximization](https://mlsanity.com/paper/2205.12381/) (2022)
- [Assistive Teaching of Motor Control Tasks to Humans](https://mlsanity.com/paper/2211.14003/) (2022)
- [Dreaming to Assist: Learning to Align with Human Objectives for Shared Control in High-Speed Racing](https://mlsanity.com/paper/2410.10062/) (2024)
- [Genie Centurion: Accelerating Scalable Real-World Robot Training with Human Rewind-and-Refine Guidance](https://mlsanity.com/paper/2505.18793/) (2025)
- [A Cognitive Framework for Delegation Between Error-Prone AI and Human Agents](https://mlsanity.com/paper/2204.02889/) (2022)
- [Proactive and Reactive Engagement of Artificial Intelligence Methods for Education: A Review](https://mlsanity.com/paper/2301.10231/) (2023)
- [Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLA](https://mlsanity.com/paper/2603.08122/) (2026)
- [Auditing Robot Learning for Safety and Compliance during Deployment](https://mlsanity.com/paper/2110.05702/) (2021)
- [Zero-Shot Assistance in Sequential Decision Problems](https://mlsanity.com/paper/2202.07364/) (2022)
- [X2T: Training an X-to-Text Typing Interface with Online Learning from User Feedback](https://mlsanity.com/paper/2203.02072/) (2022)

## Abstract (arXiv)

In shared autonomy, user input is combined with semi-autonomous control to achieve a common goal. The goal is often unknown ex-ante, so prior work enables agents to infer the goal from user input and assist with the task. Such methods tend to assume some combination of knowledge of the dynamics of the environment, the user's policy given their goal, and the set of possible goals the user might target, which limits their application to real-world scenarios. We propose a deep reinforcement learning framework for model-free shared autonomy that lifts these assumptions. We use human-in-the-loop reinforcement learning with neural network function approximation to learn an end-to-end mapping from environmental observation and user input to agent action values, with task reward as the only form of supervision. This approach poses the challenge of following user commands closely enough to provide the user with real-time action feedback and thereby ensure high-quality user input, but also deviating from the user's actions when they are suboptimal. We balance these two needs by discarding actions whose values fall below some threshold, then selecting the remaining action closest to the user's input. Controlled studies with users (n = 12) and synthetic pilots playing a video game, and a pilot study with users (n = 4) flying a real quadrotor, demonstrate the ability of our algorithm to assist users with real-time control tasks in which the agent cannot directly access the user's private information through observations, but receives a reward signal and user input that both depend on the user's intent. The agent learns to assist the user without access to this private information, implicitly inferring it from the user's input. This paper is a proof of concept that illustrates the potential for deep reinforcement learning to enable flexible and practical assistive systems.