# Dynamic Input for Deep Reinforcement Learning in Autonomous Driving

arXiv: https://arxiv.org/abs/1907.10994 · HTML: https://mlsanity.com/paper/1907.10994/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2019.

## Built on by (most cited first, 11 of 11)

- [Reward (Mis)design for Autonomous Driving](https://mlsanity.com/paper/2104.13906/) (2021)
- [A Survey of Deep RL and IL for Autonomous Driving Policy Learning](https://mlsanity.com/paper/2101.01993/) (2021)
- [Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2303.01212/) (2023)
- [Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning](https://mlsanity.com/paper/2107.07316/) (2021)
- [Cooperative Behavior Planning for Automated Driving using Graph Neural Networks](https://mlsanity.com/paper/2202.11376/) (2022)
- [Optimizing Trajectories for Highway Driving with Offline Reinforcement Learning](https://mlsanity.com/paper/2203.10949/) (2022)
- [Integration of Reinforcement Learning Based Behavior Planning With Sampling Based Motion Planning for Automated Driving](https://mlsanity.com/paper/2304.08280/) (2023)
- [Autonomous Navigation through intersections with Graph ConvolutionalNetworks and Conditional Imitation Learning for Self-driving Cars](https://mlsanity.com/paper/2102.00675/) (2021)
- [Bi-Level Optimization Augmented with Conditional Variational Autoencoder for Autonomous Driving in Dense Traffic](https://mlsanity.com/paper/2212.02224/) (2022)
- [High-level Decisions from a Safe Maneuver Catalog with Reinforcement Learning for Safe and Cooperative Automated Merging](https://mlsanity.com/paper/2107.07413/) (2021)
- [Agent-Agnostic Centralized Training for Decentralized Multi-Agent Cooperative Driving](https://mlsanity.com/paper/2403.11914/) (2024)

## Abstract (arXiv)

In many real-world decision making problems, reaching an optimal decision requires taking into account a variable number of objects around the agent. Autonomous driving is a domain in which this is especially relevant, since the number of cars surrounding the agent varies considerably over time and affects the optimal action to be taken. Classical methods that process object lists can deal with this requirement. However, to take advantage of recent high-performing methods based on deep reinforcement learning in modular pipelines, special architectures are necessary. For these, a number of options exist, but a thorough comparison of the different possibilities is missing. In this paper, we elaborate limitations of fully-connected neural networks and other established approaches like convolutional and recurrent neural networks in the context of reinforcement learning problems that have to deal with variable sized inputs. We employ the structure of Deep Sets in off-policy reinforcement learning for high-level decision making, highlight their capabilities to alleviate these limitations, and show that Deep Sets not only yield the best overall performance but also offer better generalization to unseen situations than the other approaches.