# Collision Avoidance in Pedestrian-Rich Environments with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1910.11689 · HTML: https://mlsanity.com/paper/1910.11689/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2019.

## Built on by (most cited first, 30 of 49)

- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation](https://mlsanity.com/paper/2203.15041/) (2022)
- [Principles and Guidelines for Evaluating Social Robot Navigation Algorithms](https://mlsanity.com/paper/2306.16740/) (2023)
- [No Regrets: Investigating and Improving Regret Approximations for Curriculum Discovery](https://mlsanity.com/paper/2408.15099/) (2024)
- [Review of Pedestrian Trajectory Prediction Methods: Comparing Deep Learning and Knowledge-based Approaches](https://mlsanity.com/paper/2111.06740/) (2021)
- [Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians](https://mlsanity.com/paper/2102.13073/) (2021)
- [Injecting Planning-Awareness into Prediction and Detection Evaluation](https://mlsanity.com/paper/2110.03270/) (2021)
- [A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2209.03859/) (2022)
- [Rethinking Trajectory Forecasting Evaluation](https://mlsanity.com/paper/2107.10297/) (2021)
- [Sequential Neural Barriers for Scalable Dynamic Obstacle Avoidance](https://mlsanity.com/paper/2307.03015/) (2023)
- [Nav-Q: Quantum Deep Reinforcement Learning for Collision-Free Navigation of Self-Driving Cars](https://mlsanity.com/paper/2311.12875/) (2023)
- [Robot Navigation in a Crowd by Integrating Deep Reinforcement Learning and Online Planning](https://mlsanity.com/paper/2102.13265/) (2021)
- [Towards Disturbance-Free Visual Mobile Manipulation](https://mlsanity.com/paper/2112.12612/) (2021)
- [Learning Team-Based Navigation: A Review of Deep Reinforcement Learning Techniques for Multi-Agent Pathfinding](https://mlsanity.com/paper/2308.05893/) (2023)
- [Multi-agent Soft Actor-Critic Based Hybrid Motion Planner for Mobile Robots](https://mlsanity.com/paper/2112.06594/) (2021)
- [SOCIALGYM 2.0: Simulator for Multi-Agent Social Robot Navigation in Shared Human Spaces](https://mlsanity.com/paper/2303.05584/) (2023)
- [Socially Integrated Navigation: A Social Acting Robot with Deep Reinforcement Learning](https://mlsanity.com/paper/2403.09793/) (2024)
- [Safe Deep Q-Network for Autonomous Vehicles at Unsignalized Intersection](https://mlsanity.com/paper/2106.04561/) (2021)
- [Using Collision Momentum in Deep Reinforcement Learning Based Adversarial Pedestrian Modeling](https://mlsanity.com/paper/2306.07525/) (2023)
- [Attention Graph for Multi-Robot Social Navigation with Deep Reinforcement Learning](https://mlsanity.com/paper/2401.17914/) (2024)
- [Learning-Based UAV Trajectory Optimization with Collision Avoidance and Connectivity Constraints](https://mlsanity.com/paper/2104.06256/) (2021)
- [Nearest-Neighbor-based Collision Avoidance for Quadrotors via Reinforcement Learning](https://mlsanity.com/paper/2104.14912/) (2021)
- [Spatial-temporal recurrent reinforcement learning for autonomous ships](https://mlsanity.com/paper/2211.01004/) (2022)
- [Hybrid Deep Reinforcement Learning and Planning for Safe and Comfortable Automated Driving](https://mlsanity.com/paper/2301.00650/) (2022)
- [Pedestrian Collision Avoidance for Autonomous Vehicles at Unsignalized Intersection Using Deep Q-Network](https://mlsanity.com/paper/2105.00153/) (2021)
- [Decentralized Social Navigation with Non-Cooperative Robots via Bi-Level Optimization](https://mlsanity.com/paper/2306.08815/) (2023)
- [Disentangling Uncertainty for Safe Social Navigation using Deep Reinforcement Learning](https://mlsanity.com/paper/2409.10655/) (2024)
- [Dynamic Obstacle Avoidance through Uncertainty-Based Adaptive Planning with Diffusion](https://mlsanity.com/paper/2409.16950/) (2024)
- [Scalable Task-Driven Robotic Swarm Control via Collision Avoidance and Learning Mean-Field Control](https://mlsanity.com/paper/2209.07420/) (2022)
- [Active Classification of Moving Targets with Learned Control Policies](https://mlsanity.com/paper/2212.03068/) (2022)

## Abstract (arXiv)

Collision avoidance algorithms are essential for safe and efficient robot operation among pedestrians. This work proposes using deep reinforcement (RL) learning as a framework to model the complex interactions and cooperation with nearby, decision-making agents, such as pedestrians and other robots. Existing RL-based works assume homogeneity of agent properties, use specific motion models over short timescales, or lack a principled method to handle a large, possibly varying number of agents. Therefore, this work develops an algorithm that learns collision avoidance among a variety of heterogeneous, non-communicating, dynamic agents without assuming they follow any particular behavior rules. It extends our previous work by introducing a strategy using Long Short-Term Memory (LSTM) that enables the algorithm to use observations of an arbitrary number of other agents, instead of a small, fixed number of neighbors. The proposed algorithm is shown to outperform a classical collision avoidance algorithm, another deep RL-based algorithm, and scales with the number of agents better (fewer collisions, shorter time to goal) than our previously published learning-based approach. Analysis of the LSTM provides insights into how observations of nearby agents affect the hidden state and quantifies the performance impact of various agent ordering heuristics. The learned policy generalizes to several applications beyond the training scenarios: formation control (arrangement into letters), demonstrations on a fleet of four multirotors and on a fully autonomous robotic vehicle capable of traveling at human walking speed among pedestrians.