# Robot Crowd Navigation in Dynamic Environment with Offline Reinforcement Learning

arXiv: https://arxiv.org/abs/2312.11032 · HTML: https://mlsanity.com/paper/2312.11032/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Social robot navigation, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 18 Dec 2023.

## Placed in

- Offline RL and imitation learning from data / [Social robot navigation](https://mlsanity.com/offlinerl/social-robot-navigation/)

## Builds on (most cited first, 13 of 13)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021)
- [Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning](https://mlsanity.com/paper/1910.00177/) (2019)
- [Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.07845/) (2016)
- [Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1805.01956/) (2018)
- [Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning](https://mlsanity.com/paper/1709.10082/) (2017)
- [Intention Aware Robot Crowd Navigation with Attention-Based Interaction Graph](https://mlsanity.com/paper/2203.01821/) (2022)
- [Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning](https://mlsanity.com/paper/2011.04820/) (2020)
- [DenseCAvoid: Real-time Navigation in Dense Crowds using Anticipatory Behaviors](https://mlsanity.com/paper/2002.03038/) (2020)
- [Deep-Learned Collision Avoidance Policy for Distributed Multi-Agent Navigation](https://mlsanity.com/paper/1609.06838/) (2016)
- [NaviSTAR: Socially Aware Robot Navigation with Hybrid Spatio-Temporal Graph Transformer and Preference Learning](https://mlsanity.com/paper/2304.05979/) (2023)
- [Robot Navigation in a Crowd by Integrating Deep Reinforcement Learning and Online Planning](https://mlsanity.com/paper/2102.13265/) (2021)

## Abstract (arXiv)

Robot crowd navigation has been gaining increasing attention and popularity in various practical applications. In existing research, deep reinforcement learning has been applied to robot crowd navigation by training policies in an online mode. However, this inevitably leads to unsafe exploration, and consequently causes low sampling efficiency during pedestrian-robot interaction. To this end, we propose an offline reinforcement learning based robot crowd navigation algorithm by utilizing pre-collected crowd navigation experience. Specifically, this algorithm integrates a spatial-temporal state into implicit Q-Learning to avoid querying out-of-distribution robot actions of the pre-collected experience, while capturing spatial-temporal features from the offline pedestrian-robot interactions. Experimental results demonstrate that the proposed algorithm outperforms the state-of-the-art methods by means of qualitative and quantitative analysis.