# Planning with Occluded Traffic Agents using Bi-Level Variational Occlusion Models

arXiv: https://arxiv.org/abs/2210.14584 · HTML: https://mlsanity.com/paper/2210.14584/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 26 Oct 2022.

## Builds on (most cited first, 11 of 11)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Learning to Look around Objects for Top-View Representations of Outdoor Scenes](https://mlsanity.com/paper/1803.10870/) (2018)
- [Safe Occlusion-aware Autonomous Driving via Game-Theoretic Active Perception](https://mlsanity.com/paper/2105.08169/) (2021)
- [Interpretable Goal Recognition in the Presence of Occluded Factors for Autonomous Vehicles](https://mlsanity.com/paper/2108.02530/) (2021)
- [Tackling Occlusions & Limited Sensor Range with Set-based Safety Verification](https://mlsanity.com/paper/1807.01262/) (2018)
- [Seeing Behind Things: Extending Semantic Segmentation to Occluded Regions](https://mlsanity.com/paper/1906.02885/) (2019)
- [Multi-Agent Variational Occlusion Inference Using People as Sensors](https://mlsanity.com/paper/2109.02173/) (2021)
- [Planning Paths Through Unknown Space by Imagining What Lies Therein](https://mlsanity.com/paper/2011.07316/) (2020)
- [HMPO: Human Motion Prediction in Occluded Environments for Safe Motion Planning](https://mlsanity.com/paper/2006.00424/) (2020)

## Built on by (most cited first, 5 of 5)

- [trajdata: A Unified Interface to Multiple Human Trajectory Datasets](https://mlsanity.com/paper/2307.13924/) (2023)
- [Scene Informer: Anchor-based Occlusion Inference and Trajectory Prediction in Partially Observable Environments](https://mlsanity.com/paper/2309.13893/) (2023)
- [Towards Vision Zero: The TUM Traffic Accid3nD Dataset](https://mlsanity.com/paper/2503.12095/) (2025)
- [Enhancing Highway Safety: Accident Detection on the A9 Test Stretch Using Roadside Sensors](https://mlsanity.com/paper/2502.00402/) (2025)
- [Safety-Critical Learning for Long-Tail Events: The TUM Traffic Accident Dataset](https://mlsanity.com/paper/2508.14567/) (2025)

## Abstract (arXiv)

Reasoning with occluded traffic agents is a significant open challenge for planning for autonomous vehicles. Recent deep learning models have shown impressive results for predicting occluded agents based on the behaviour of nearby visible agents; however, as we show in experiments, these models are difficult to integrate into downstream planning. To this end, we propose Bi-level Variational Occlusion Models (BiVO), a two-step generative model that first predicts likely locations of occluded agents, and then generates likely trajectories for the occluded agents. In contrast to existing methods, BiVO outputs a trajectory distribution which can then be sampled from and integrated into standard downstream planning. We evaluate the method in closed-loop replay simulation using the real-world nuScenes dataset. Our results suggest that BiVO can successfully learn to predict occluded agent trajectories, and these predictions lead to better subsequent motion plans in critical scenarios.