# Decentralized cooperative perception for autonomous vehicles: Learning to value the unknown

arXiv: https://arxiv.org/abs/2301.01250 · HTML: https://mlsanity.com/paper/2301.01250/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 12 Dec 2022.

## Placed in

- End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/)

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction](https://mlsanity.com/paper/2008.07519/) (2020)
- [F-Cooper: Feature based Cooperative Perception for Autonomous Vehicle Edge Computing System Using 3D Point Clouds](https://mlsanity.com/paper/1909.06459/) (2019)
- [Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review](https://mlsanity.com/paper/2005.09830/) (2020)
- [Shaping Belief States with Generative Environment Models for RL](https://mlsanity.com/paper/1906.09237/) (2019)
- [Temporal Difference Variational Auto-Encoder](https://mlsanity.com/paper/1806.03107/) (2018)
- [Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling](https://mlsanity.com/paper/1705.08781/) (2017)
- [Cooperative Perception with Deep Reinforcement Learning for Connected Vehicles](https://mlsanity.com/paper/2004.10927/) (2020)
- [Planning Beyond the Sensing Horizon Using a Learned Context](https://mlsanity.com/paper/1908.09171/) (2019)
- [Evidential Occupancy Grid Map Augmentation using Deep Learning](https://mlsanity.com/paper/1801.05297/) (2018)
- [Focal points and their implications for M"obius Transforms and Dempster-Shafer Theory](https://mlsanity.com/paper/2011.06549/) (2020)

## Abstract (arXiv)

Recently, we have been witnesses of accidents involving autonomous vehicles and their lack of sufficient information. One way to tackle this issue is to benefit from the perception of different view points, namely cooperative perception. We propose here a decentralized collaboration, i.e. peer-to-peer, in which the agents are active in their quest for full perception by asking for specific areas in their surroundings on which they would like to know more. Ultimately, we want to optimize a trade-off between the maximization of knowledge about moving objects and the minimization of the total volume of information received from others, to limit communication costs and message processing time. For this, we propose a way to learn a communication policy that reverses the usual communication paradigm by only requesting from other vehicles what is unknown to the ego-vehicle, instead of filtering on the sender side. We tested three different generative models to be taken as base for a Deep Reinforcement Learning (DRL) algorithm, and compared them to a broadcasting policy and a policy randomly selecting areas. In particular, we propose Locally Predictable VAE (LP-VAE), which appears to be producing better belief states for predictions than state-of-the-art models, both as a standalone model and in the context of DRL. Experiments were conducted in the driving simulator CARLA. Our best models reached on average a gain of 25% of the total complementary information, while only requesting about 5% of the ego-vehicle's perceptual field. This trade-off is adjustable through the interpretable hyperparameters of our reward function.