Quantifying Assistive Robustness Via the Natural-Adversarial Frontier
Not on the map. Why?
Abstract (arXiv)
Our ultimate goal is to build robust policies for robots that assist people. What makes this hard is that people can behave unexpectedly at test time, potentially interacting with the robot outside its training distribution and leading to failures. Even just measuring robustness is a challenge. Adversarial perturbations are the default, but they can paint the wrong picture: they can correspond to human motions that are unlikely to occur during natural interactions with people. A robot policy might fail under small adversarial perturbations but work under large natural perturbations. We propose that capturing robustness in these interactive settings requires constructing and analyzing the entire natural-adversarial frontier: the Pareto-frontier of human policies that are the best trade-offs between naturalness and low robot performance. We introduce RIGID, a method for constructing this frontier by training adversarial human policies that trade off between minimizing robot reward and acting human-like (as measured by a discriminator). On an Assistive Gym task, we use RIGID to analyze the performance of standard collaborative Reinforcement Learning, as well as the performance of existing methods meant to increase robustness. We also compare the frontier RIGID identifies with the failures identified in expert adversarial interaction, and with naturally-occurring failures during user interaction. Overall, we find evidence that RIGID can provide a meaningful measure of robustness predictive of deployment performance, and uncover failure cases in human-robot interaction that are difficult to find manually. https://ood-human.github.io.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 42 of its 61 references that are in the map. First posted on arXiv 16 Oct 2023.
Builds on
42 of 61 references.
References · 42
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2013Intriguing properties of neural networkscited by 5,155
- 2015Trust Region Policy Optimizationcited by 2,580
- 2016Adversarial examples in the physical worldcited by 1,769
- 2016Generative Adversarial Imitation Learningcited by 1,597
- 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
- 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
- 2016Least Squares Generative Adversarial Networkscited by 1,001
- 2016Adversarial Machine Learning at Scalecited by 936
- 2017Ensemble Adversarial Training: Attacks and Defensescited by 840
30 more
- 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 536
- 2017Robust Adversarial Reinforcement Learningcited by 451
- 2019On Evaluating Adversarial Robustnesscited by 447
- 2015Illuminating search spaces by mapping elitescited by 425
- 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
- 2017Adversarial Attacks on Neural Network Policiescited by 320
- 2018Certified Defenses against Adversarial Examplescited by 306
- 2016Cooperative Inverse Reinforcement Learningcited by 300
- 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 293
- 2017PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examplescited by 244
- 2018Adversarial Risk and the Dangers of Evaluating Against Weak Attackscited by 221
- 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 214
- 2018A Study on Overfitting in Deep Reinforcement Learningcited by 174
- 2014Towards Deep Neural Network Architectures Robust to Adversarial Examplescited by 146
- 2021Collaborating with Humans without Human Datacited by 142
- 2022Adversarial Motion Priors Make Good Substitutes for Complex Reward Functionscited by 89
- 2021Differentiable Quality Diversitycited by 72
- 2022Unpacking Reward Shaping: Understanding the Benefits of Reward Engineering on Sample Complexitycited by 62
- 2015Distributional Smoothing with Virtual Adversarial Trainingcited by 61
- 2022Causal Confusion and Reward Misidentification in Preference-Based Reward Learningcited by 53
- 2019Assistive Gym: A Physics Simulation Framework for Assistive Roboticscited by 44
- 2017Variational Inference using Implicit Distributionscited by 39
- 2022The Boltzmann Policy Distribution: Accounting for Systematic Suboptimality in Human Modelscited by 31
- 2022Understanding the Complexity Gains of Single-Task RL with a Curriculumcited by 17
- 2019Learning to Collaborate from Simulation for Robot-Assisted Dressingcited by 16
- 2022Learning Representations that Enable Generalization in Assistive Taskscited by 13
- 2020Assistive VR Gym: Interactions with Real People to Improve Virtual Assistive Robotscited by 5
- 2021Assisted Robust Reward Designcited by 5
- 2019Disturbance Observer-based Robust Control and Its Applications: 35th Anniversary Overviewcited by 4
- 2022Reducing Exploitability with Population Based Trainingcited by 4
Built on by
Citing papers per year: 2025: 1.
1 paper.