Online Behavior Modification for Expressive User Control of RL-Trained Robots
Not on the map. Why?
Abstract (arXiv)
Reinforcement Learning (RL) is an effective method for robots to learn tasks. However, in typical RL, end-users have little to no control over how the robot does the task after the robot has been deployed. To address this, we introduce the idea of online behavior modification, a paradigm in which users have control over behavior features of a robot in real time as it autonomously completes a task using an RL-trained policy. To show the value of this user-centered formulation for human-robot interaction, we present a behavior diversity based algorithm, Adjustable Control Of RL Dynamics (ACORD), and demonstrate its applicability to online behavior modification in simulation and a user study. In the study (n=23) users adjust the style of paintings as a robot traces a shape autonomously. We compare ACORD to RL and Shared Autonomy (SA), and show ACORD affords user-preferred levels of control and expression, comparable to SA, but with the potential for autonomous execution and robustness of RL.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 30 of its 55 references that are in the map. First posted on arXiv 15 Aug 2024.
Builds on
30 of 55 references.
References · 30
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
- 2017Hindsight Experience Replaycited by 1,096
- 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
- 2017Safe Reinforcement Learning via Shieldingcited by 427
- 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
- 2021A Survey of Human-in-the-loop for Machine Learningcited by 214
- 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
- 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
- 2022Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behaviorcited by 113
- 2020One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RLcited by 79
18 more
- 2020Learning Reward Functions from Diverse Sources of Human Feedback: Optimally Integrating Demonstrations and Preferencescited by 71
- 2021Differentiable Quality Diversitycited by 71
- 2018Safe Reinforcement Learning with Model Uncertainty Estimatescited by 70
- 2020Active Preference-Based Gaussian Process Regression for Reward Learningcited by 70
- 2018Shared Autonomy via Deep Reinforcement Learningcited by 65
- 2019Autonomous skill discovery with Quality-Diversity and Unsupervised Descriptorscited by 57
- 2015Shared Autonomy via Hindsight Optimizationcited by 46
- 2021A Survey on Interactive Reinforcement Learning: Design Principles and Open Challengescited by 41
- 2017Shared Autonomy via Hindsight Optimization for Teleoperation and Teamingcited by 39
- 2022Approximating Gradients for Differentiable Quality Diversity in Reinforcement Learningcited by 37
- 2023pyribs: A Bare-Bones Python Library for Quality Diversity Optimizationcited by 29
- 2020Feature Expansive Reward Learning: Rethinking Human Inputcited by 25
- 2021Discovering Diverse Solutions in Deep Reinforcement Learning by Maximizing State-Action-Based Mutual Informationcited by 18
- 2019Adversarial Skill Networks: Unsupervised Robot Skill Learning from Videocited by 17
- 2018Learning from Richer Human Guidance: Augmenting Comparison-Based Learning with Feature Queriescited by 16
- 2021Situational Confidence Assistance for Lifelong Shared Autonomycited by 15
- 2018Simplifying Reward Design through Divide-and-Conquercited by 12
- 2018HARMONIC: A Multimodal Dataset of Assistive Human-Robot Collaborationcited by 10
Built on by
Not cited yet.