# Towards Learning Generalizable Driving Policies from Restricted Latent Representations

arXiv: https://arxiv.org/abs/2111.03688 · HTML: https://mlsanity.com/paper/2111.03688/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 29 of its 45 references that are in the map.
- First posted on arXiv 5 Nov 2021.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)

## Builds on (most cited first, 29 of 29)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Cognitive Mapping and Planning for Visual Navigation](https://mlsanity.com/paper/1702.03920/) (2017)
- [Recent Advances in Autoencoder-Based Representation Learning](https://mlsanity.com/paper/1812.05069/) (2018)
- [Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learning](https://mlsanity.com/paper/2001.08726/) (2020)
- [Model-free Deep Reinforcement Learning for Urban Autonomous Driving](https://mlsanity.com/paper/1904.09503/) (2019)
- [Learning Latent Representations to Influence Multi-Agent Interaction](https://mlsanity.com/paper/2011.06619/) (2020)
- [Reinforcement Learning for Autonomous Driving with Latent State Inference and Spatial-Temporal Relationships](https://mlsanity.com/paper/2011.04251/) (2020)
- [Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling](https://mlsanity.com/paper/1705.08781/) (2017)
- [Social Coordination and Altruism in Autonomous Driving](https://mlsanity.com/paper/2107.00200/) (2021)
- [Optimization for Reinforcement Learning: From Single Agent to Cooperative Agents](https://mlsanity.com/paper/1912.00498/) (2019)
- [Formulation of Deep Reinforcement Learning Architecture Toward Autonomous Driving for On-Ramp Merge](https://mlsanity.com/paper/1709.02066/) (2017)
- [Cooperative Autonomous Vehicles that Sympathize with Human Drivers](https://mlsanity.com/paper/2107.00898/) (2021)
- [Learning Latent Actions to Control Assistive Robots](https://mlsanity.com/paper/2107.02907/) (2021)
- [Controlling Steering Angle for Cooperative Self-driving Vehicles utilizing CNN and LSTM-based Deep Networks](https://mlsanity.com/paper/1904.04375/) (2019)
- [Learning Visually Guided Latent Actions for Assistive Teleoperation](https://mlsanity.com/paper/2105.00580/) (2021)
- [End-to-end Autonomous Driving Perception with Sequential Latent Representation Learning](https://mlsanity.com/paper/2003.12464/) (2020)
- [Altruistic Maneuver Planning for Cooperative Autonomous Vehicles Using Multi-agent Advantage Actor-Critic](https://mlsanity.com/paper/2107.05664/) (2021)
- [Approximate Robust Control of Uncertain Dynamical Systems](https://mlsanity.com/paper/1903.00220/) (2019)
- [Spatio-temporal Dynamics of Cellular V2X Communication in Dense Vehicular Networks](https://mlsanity.com/paper/1906.08634/) (2019)
- [A Maneuver-based Urban Driving Dataset and Model for Cooperative Vehicle Applications](https://mlsanity.com/paper/2005.13781/) (2020)
- [Multiple Access in Cellular V2X: Performance Analysis in Highly Congested Vehicular Networks](https://mlsanity.com/paper/1809.02678/) (2018)
- [Real-Time Hardware-In-the-Loop Emulation Framework for DSRC-based Connected Vehicle Applications](https://mlsanity.com/paper/1905.09267/) (2019)
- [V2X System Architecture Utilizing Hybrid Gaussian Process-based Model Structures](https://mlsanity.com/paper/1903.01576/) (2019)
- [On the Road with 16 Neurons: Mental Imagery with Bio-inspired Deep Neural Networks](https://mlsanity.com/paper/2003.08745/) (2020)
- [A Stochastic Hybrid Framework for Driver Behavior Modeling Based on Hierarchical Dirichlet Process](https://mlsanity.com/paper/1711.05655/) (2017)
- [A Driver Behavior Modeling Structure Based on Non-parametric Bayesian Stochastic Hybrid Architecture](https://mlsanity.com/paper/1807.03919/) (2018)
- [Analysis of Distributed Congestion Control in Cellular Vehicle-to-everything Networks](https://mlsanity.com/paper/1904.00071/) (2019)
- [Performance Analysis of Cellular-V2X with Adaptive and Selective Power Control](https://mlsanity.com/paper/2008.03453/) (2020)
- [Efficient Latent Representations using Multiple Tasks for Autonomous Driving](https://mlsanity.com/paper/2003.00695/) (2020)

## Built on by (most cited first, 4 of 4)

- [Robustness and Adaptability of Reinforcement Learning based Cooperative Autonomous Driving in Mixed-autonomy Traffic](https://mlsanity.com/paper/2202.00881/) (2022)
- [Exploring the trade off between human driving imitation and safety for traffic simulation](https://mlsanity.com/paper/2208.04803/) (2022)
- [Deep Occupancy-Predictive Representations for Autonomous Driving](https://mlsanity.com/paper/2303.04218/) (2023)
- [Context-Aware Target Classification with Hybrid Gaussian Process prediction for Cooperative Vehicle Safety systems](https://mlsanity.com/paper/2212.12819/) (2022)

## Abstract (arXiv)

Training intelligent agents that can drive autonomously in various urban and highway scenarios has been a hot topic in the robotics society within the last decades. However, the diversity of driving environments in terms of road topology and positioning of the neighboring vehicles makes this problem very challenging. It goes without saying that although scenario-specific driving policies for autonomous driving are promising and can improve transportation safety and efficiency, they are clearly not a universal scalable solution. Instead, we seek decision-making schemes and driving policies that can generalize to novel and unseen environments. In this work, we capitalize on the key idea that human drivers learn abstract representations of their surroundings that are fairly similar among various driving scenarios and environments. Through these representations, human drivers are able to quickly adapt to novel environments and drive in unseen conditions. Formally, through imposing an information bottleneck, we extract a latent representation that minimizes the textitdistance -- a quantification that we introduce to gauge the similarity among different driving configurations -- between driving scenarios. This latent space is then employed as the input to a Q-learning module to learn generalizable driving policies. Our experiments revealed that, using this latent representation can reduce the number of crashes to about half.