An Online Data-Driven Emergency-Response Method for Autonomous Agents in Unforeseen Situations
Not on the map. Why?
Abstract (arXiv)
Reinforcement learning agents perform well when presented with inputs within the distribution of those encountered during training. However, they are unable to respond effectively when faced with novel, out-of-distribution events, until they have undergone additional training. This paper presents an online, data-driven, emergency-response method that aims to provide autonomous agents the ability to react to unexpected situations that are very different from those it has been trained or designed to address. In such situations, learned policies cannot be expected to perform appropriately since the observations obtained in these novel situations would fall outside the distribution of inputs that the agent has been optimized to handle. The proposed approach devises a customized response to the unforeseen situation sequentially, by selecting actions that minimize the rate of increase of the reconstruction error from a variational auto-encoder. This optimization is achieved online in a data-efficient manner (on the order of 30 data-points) using a modified Bayesian optimization procedure. We demonstrate the potential of this approach in a simulated 3D car driving scenario, in which the agent devises a response in under 2 seconds to avoid collisions with objects it has not seen during training.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 16 of its 36 references that are in the map. First posted on arXiv 17 Dec 2021.
Builds on
16 of 36 references.
References · 16
- 2013Intriguing properties of neural networkscited by 5,155
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
- 2018World Modelscited by 1,513
- 2017Constrained Policy Optimizationcited by 983
- 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 803
- 2017A Brief Survey of Deep Reinforcement Learningcited by 483
- 2018Reward Constrained Policy Optimizationcited by 417
- 2017Variational Continual Learningcited by 339
- 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
- 2018An Introduction to Deep Reinforcement Learningcited by 206
- 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
4 more
- 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 163
- 2017Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learningcited by 90
- 2017Virtual to Real Reinforcement Learning for Autonomous Drivingcited by 80
- 2019S-TRIGGER: Continual State Representation Learning via Self-Triggered Generative Replaycited by 7
Built on by
Citing papers per year: 2023: 1.
1 paper.