# Realtime Collision Avoidance for Mobile Robots in Dense Crowds using Implicit Multi-sensor Fusion and Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2004.03089 · HTML: https://mlsanity.com/paper/2004.03089/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2020.

## Built on by (most cited first, 4 of 4)

- [Real-Time Monocular Human Depth Estimation and Segmentation on Embedded Systems](https://mlsanity.com/paper/2108.10506/) (2021)
- [Intention-Aware Navigation in Crowds with Extended-Space POMDP Planning](https://mlsanity.com/paper/2206.10028/) (2022)
- [Distributed Swarm Collision Avoidance Based on Angular Calculations](https://mlsanity.com/paper/2108.12934/) (2021)
- [Learned Tree Search for Long-Horizon Social Robot Navigation in Shared Airspace](https://mlsanity.com/paper/2304.01428/) (2023)

## Abstract (arXiv)

We present a novel learning-based collision avoidance algorithm, CrowdSteer, for mobile robots operating in dense and crowded environments. Our approach is end-to-end and uses multiple perception sensors such as a 2-D lidar along with a depth camera to sense surrounding dynamic agents and compute collision-free velocities. Our training approach is based on the sim-to-real paradigm and uses high fidelity 3-D simulations of pedestrians and the environment to train a policy using Proximal Policy Optimization (PPO). We show that our learned navigation model is directly transferable to previously unseen virtual and dense real-world environments. We have integrated our algorithm with differential drive robots and evaluated its performance in narrow scenarios such as dense crowds, narrow corridors, T-junctions, L-junctions, etc. In practice, our approach can perform real-time collision avoidance and generate smooth trajectories in such complex scenarios. We also compare the performance with prior methods based on metrics such as trajectory length, mean time to goal, success rate, and smoothness and observe considerable improvement.