# Social Occlusion Inference with Vectorized Representation for Autonomous Driving

arXiv: https://arxiv.org/abs/2303.10385 · HTML: https://mlsanity.com/paper/2303.10385/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 18 Mar 2023.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps](https://mlsanity.com/paper/1910.03088/) (2019)
- [Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks](https://mlsanity.com/paper/1907.03395/) (2019)
- [Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2005.08514/) (2020)
- [VAIN: Attentional Multi-agent Predictive Modeling](https://mlsanity.com/paper/1706.06122/) (2017)
- [Safety-aware Motion Prediction with Unseen Vehicles for Autonomous Driving](https://mlsanity.com/paper/2109.01510/) (2021)
- [Multi-Agent Variational Occlusion Inference Using People as Sensors](https://mlsanity.com/paper/2109.02173/) (2021)
- [People as Sensors: Imputing Maps from Human Actions](https://mlsanity.com/paper/1711.01022/) (2017)
- [Limited Visibility and Uncertainty Aware Motion Planning for Automated Driving](https://mlsanity.com/paper/1810.13001/) (2018)
- [Motion Planning for Autonomous Vehicles in the Presence of Uncertainty Using Reinforcement Learning](https://mlsanity.com/paper/2110.00640/) (2021)
- [Behavior Planning of Autonomous Cars with Social Perception](https://mlsanity.com/paper/1905.00988/) (2019)
- [Dynamic Hilbert Maps: Real-Time Occupancy Predictions in Changing Environment](https://mlsanity.com/paper/1912.02149/) (2019)

## Built on by (most cited first, 1 of 1)

- [Towards Developing Socially Compliant Automated Vehicles: Advances, Expert Insights, and A Conceptual Framework](https://mlsanity.com/paper/2501.06089/) (2025)

## Abstract (arXiv)

Autonomous vehicles must be capable of handling the occlusion of the environment to ensure safe and efficient driving. In urban environment, occlusion often arises due to other vehicles obscuring the perception of the ego vehicle. Since the occlusion condition can impact the trajectories of vehicles, the behavior of other vehicles is helpful in making inferences about the occlusion as a remedy for perceptual deficiencies. This paper introduces a novel social occlusion inference approach that learns a mapping from agent trajectories and scene context to an occupancy grid map (OGM) representing the view of ego vehicle. Specially, vectorized features are encoded through the polyline encoder to aggregate features of vectors into features of polylines. A transformer module is then utilized to model the high-order interactions of polylines. Importantly, occlusion queries are proposed to fuse polyline features and generate the OGM without the input of visual modality. To verify the performance of vectorized representation, we design a baseline based on a fully transformer encoder-decoder architecture mapping the OGM with occlusion and historical trajectories information to the ground truth OGM. We evaluate our approach on an unsignalized intersection in the INTERACTION dataset, which outperforms the state-of-the-art results.