# Occlusion-Aware Crowd Navigation Using People as Sensors

arXiv: https://arxiv.org/abs/2210.00552 · HTML: https://mlsanity.com/paper/2210.00552/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 43 references that are in the map.
- First posted on arXiv 2 Oct 2022.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Self-Supervised Visual Planning with Temporal Skip Connections](https://mlsanity.com/paper/1710.05268/) (2017)
- [Socially Aware Motion Planning with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.08862/) (2017)
- [Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.07845/) (2016)
- [Relational Graph Learning for Crowd Navigation](https://mlsanity.com/paper/1909.13165/) (2019)
- [Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning](https://mlsanity.com/paper/2011.04820/) (2020)
- [DualSMC: Tunneling Differentiable Filtering and Planning under Continuous POMDPs](https://mlsanity.com/paper/1909.13003/) (2019)
- [Learning-based 3D Occupancy Prediction for Autonomous Navigation in Occluded Environments](https://mlsanity.com/paper/2011.03981/) (2020)
- [Safe Reinforcement Learning with Scene Decomposition for Navigating Complex Urban Environments](https://mlsanity.com/paper/1904.11483/) (2019)
- [Dynamic Environment Prediction in Urban Scenes using Recurrent Representation Learning](https://mlsanity.com/paper/1904.12374/) (2019)
- [Double-Prong ConvLSTM for Spatiotemporal Occupancy Prediction in Dynamic Environments](https://mlsanity.com/paper/2011.09045/) (2020)
- [Attention Augmented ConvLSTM for Environment Prediction](https://mlsanity.com/paper/2010.09662/) (2020)
- [Belief State Planning for Autonomously Navigating Urban Intersections](https://mlsanity.com/paper/1704.04322/) (2017)
- [Pedestrian Prediction by Planning using Deep Neural Networks](https://mlsanity.com/paper/1706.05904/) (2017)
- [Seeing Behind Things: Extending Semantic Segmentation to Occluded Regions](https://mlsanity.com/paper/1906.02885/) (2019)
- [ROLL: Visual Self-Supervised Reinforcement Learning with Object Reasoning](https://mlsanity.com/paper/2011.06777/) (2020)
- [Multi-Agent Variational Occlusion Inference Using People as Sensors](https://mlsanity.com/paper/2109.02173/) (2021)
- [People as Sensors: Imputing Maps from Human Actions](https://mlsanity.com/paper/1711.01022/) (2017)
- [Behavior Planning of Autonomous Cars with Social Perception](https://mlsanity.com/paper/1905.00988/) (2019)
- [Semantic Foreground Inpainting from Weak Supervision](https://mlsanity.com/paper/1909.04564/) (2019)
- [What we see and What we don't see: Imputing Occluded Crowd Structures from Robot Sensing](https://mlsanity.com/paper/2109.08494/) (2021)

## Built on by (most cited first, 9 of 9)

- [SACSoN: Scalable Autonomous Control for Social Navigation](https://mlsanity.com/paper/2306.01874/) (2023)
- [Scene Informer: Anchor-based Occlusion Inference and Trajectory Prediction in Partially Observable Environments](https://mlsanity.com/paper/2309.13893/) (2023)
- [HEIGHT: Heterogeneous Interaction Graph Transformer for Robot Navigation in Crowded and Constrained Environments](https://mlsanity.com/paper/2411.12150/) (2024)
- [SELFI: Autonomous Self-Improvement with Reinforcement Learning for Social Navigation](https://mlsanity.com/paper/2403.00991/) (2024)
- [Disentangling Uncertainty for Safe Social Navigation using Deep Reinforcement Learning](https://mlsanity.com/paper/2409.10655/) (2024)
- [Socially aware navigation for mobile robots: a survey on deep reinforcement learning approaches](https://mlsanity.com/paper/2512.00049/) (2025)
- [Belief Aided Navigation using Bayesian Reinforcement Learning for Avoiding Humans in Blind Spots](https://mlsanity.com/paper/2403.10105/) (2024)
- [Structured Graph Network for Constrained Robot Crowd Navigation with Low Fidelity Simulation](https://mlsanity.com/paper/2405.16830/) (2024)
- [SPLC: Social Preference Learning for Crowd Robot Navigation](https://mlsanity.com/paper/2607.01925/) (2026)

## Abstract (arXiv)

Autonomous navigation in crowded spaces poses a challenge for mobile robots due to the highly dynamic, partially observable environment. Occlusions are highly prevalent in such settings due to a limited sensor field of view and obstructing human agents. Previous work has shown that observed interactive behaviors of human agents can be used to estimate potential obstacles despite occlusions. We propose integrating such social inference techniques into the planning pipeline. We use a variational autoencoder with a specially designed loss function to learn representations that are meaningful for occlusion inference. This work adopts a deep reinforcement learning approach to incorporate the learned representation for occlusion-aware planning. In simulation, our occlusion-aware policy achieves comparable collision avoidance performance to fully observable navigation by estimating agents in occluded spaces. We demonstrate successful policy transfer from simulation to the real-world Turtlebot 2i. To the best of our knowledge, this work is the first to use social occlusion inference for crowd navigation.