# Robust Deep Reinforcement Learning for Security and Safety in Autonomous Vehicle Systems

arXiv: https://arxiv.org/abs/1805.00983 · HTML: https://mlsanity.com/paper/1805.00983/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2018.

## Built on by (most cited first, 17 of 17)

- [Reinforcement Learning for Feedback-Enabled Cyber Resilience](https://mlsanity.com/paper/2107.00783/) (2021)
- [Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPS](https://mlsanity.com/paper/2102.07244/) (2021)
- [A Survey of Deep Reinforcement Learning Algorithms for Motion Planning and Control of Autonomous Vehicles](https://mlsanity.com/paper/2105.14218/) (2021)
- [Machine Learning for Security in Vehicular Networks: A Comprehensive Survey](https://mlsanity.com/paper/2105.15035/) (2021)
- [Automating Privilege Escalation with Deep Reinforcement Learning](https://mlsanity.com/paper/2110.01362/) (2021)
- [DETERRENT: Detecting Trojans using Reinforcement Learning](https://mlsanity.com/paper/2208.12878/) (2022)
- [Disturbing Reinforcement Learning Agents with Corrupted Rewards](https://mlsanity.com/paper/2102.06587/) (2021)
- [Traffic-Aware Autonomous Driving with Differentiable Traffic Simulation](https://mlsanity.com/paper/2210.03772/) (2022)
- [SoK: Adversarial Machine Learning Attacks and Defences in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2301.04299/) (2023)
- [Policy Distillation with Selective Input Gradient Regularization for Efficient Interpretability](https://mlsanity.com/paper/2205.08685/) (2022)
- [Human-Aware Robot Navigation via Reinforcement Learning with Hindsight Experience Replay and Curriculum Learning](https://mlsanity.com/paper/2110.04564/) (2021)
- [A Safety-Critical Decision Making and Control Framework Combining Machine Learning and Rule-based Algorithms](https://mlsanity.com/paper/2201.12819/) (2022)
- [AI-Powered CPS-Enabled Vulnerable-User-Aware Urban Transportation Digital Twin: Methods and Applications](https://mlsanity.com/paper/2501.10396/) (2024)
- [Safeguarding connected autonomous vehicle communication: Protocols, intra- and inter-vehicular attacks and defenses](https://mlsanity.com/paper/2502.04201/) (2025)
- [An RL-Based Adaptive Detection Strategy to Secure Cyber-Physical Systems](https://mlsanity.com/paper/2103.02872/) (2021)
- [Developing a Compressed Object Detection Model based on YOLOv4 for Deployment on Embedded GPU Platform of Autonomous System](https://mlsanity.com/paper/2108.00392/) (2021)
- [Convergence of Communications, Control, and Machine Learning for Secure and Autonomous Vehicle Navigation](https://mlsanity.com/paper/2307.02663/) (2023)

## Abstract (arXiv)

To operate effectively in tomorrow's smart cities, autonomous vehicles (AVs) must rely on intra-vehicle sensors such as camera and radar as well as inter-vehicle communication. Such dependence on sensors and communication links exposes AVs to cyber-physical (CP) attacks by adversaries that seek to take control of the AVs by manipulating their data. Thus, to ensure safe and optimal AV dynamics control, the data processing functions at AVs must be robust to such CP attacks. To this end, in this paper, the state estimation process for monitoring AV dynamics, in presence of CP attacks, is analyzed and a novel adversarial deep reinforcement learning (RL) algorithm is proposed to maximize the robustness of AV dynamics control to CP attacks. The attacker's action and the AV's reaction to CP attacks are studied in a game-theoretic framework. In the formulated game, the attacker seeks to inject faulty data to AV sensor readings so as to manipulate the inter-vehicle optimal safe spacing and potentially increase the risk of AV accidents or reduce the vehicle flow on the roads. Meanwhile, the AV, acting as a defender, seeks to minimize the deviations of spacing so as to ensure robustness to the attacker's actions. Since the AV has no information about the attacker's action and due to the infinite possibilities for data value manipulations, the outcome of the players' past interactions are fed to long-short term memory (LSTM) blocks. Each player's LSTM block learns the expected spacing deviation resulting from its own action and feeds it to its RL algorithm. Then, the the attacker's RL algorithm chooses the action which maximizes the spacing deviation, while the AV's RL algorithm tries to find the optimal action that minimizes such deviation.