# Improving the Generalization of Unseen Crowd Behaviors for Reinforcement Learning based Local Motion Planners

arXiv: https://arxiv.org/abs/2410.12232 · HTML: https://mlsanity.com/paper/2410.12232/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 25 of its 41 references that are in the map.
- First posted on arXiv 16 Oct 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.08839/) (2020)
- [Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learning](https://mlsanity.com/paper/1809.08835/) (2018)
- [Socially Aware Motion Planning with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.08862/) (2017)
- [Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.07845/) (2016)
- [Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1805.01956/) (2018)
- [Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation](https://mlsanity.com/paper/1703.00420/) (2017)
- [Celebrating Diversity in Shared Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2106.02195/) (2021)
- [Social diversity and social preferences in mixed-motive reinforcement learning](https://mlsanity.com/paper/2002.02325/) (2020)
- [Reinforced Imitation: Sample Efficient Deep Reinforcement Learning for Map-less Navigation by Leveraging Prior Demonstrations](https://mlsanity.com/paper/1805.07095/) (2018)
- [Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians](https://mlsanity.com/paper/2102.13073/) (2021)
- [DualSMC: Tunneling Differentiable Filtering and Planning under Continuous POMDPs](https://mlsanity.com/paper/1909.13003/) (2019)
- [Multi-agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learning](https://mlsanity.com/paper/2001.06627/) (2020)
- [Learning Resilient Behaviors for Navigation Under Uncertainty](https://mlsanity.com/paper/1910.09998/) (2019)
- [Mapless Navigation among Dynamics with Social-safety-awareness: a reinforcement learning approach from 2D laser scans](https://mlsanity.com/paper/1911.03074/) (2019)
- [Arena-Bench: A Benchmarking Suite for Obstacle Avoidance Approaches in Highly Dynamic Environments](https://mlsanity.com/paper/2206.05728/) (2022)
- [Getting Robots Unfrozen and Unlost in Dense Pedestrian Crowds](https://mlsanity.com/paper/1810.00352/) (2018)
- [DeepMNavigate: Deep Reinforced Multi-Robot Navigation Unifying Local & Global Collision Avoidance](https://mlsanity.com/paper/1910.09441/) (2019)
- [Learning World Transition Model for Socially Aware Robot Navigation](https://mlsanity.com/paper/2011.03922/) (2020)

## Abstract (arXiv)

Deploying a safe mobile robot policy in scenarios with human pedestrians is challenging due to their unpredictable movements. Current Reinforcement Learning-based motion planners rely on a single policy to simulate pedestrian movements and could suffer from the over-fitting issue. Alternatively, framing the collision avoidance problem as a multi-agent framework, where agents generate dynamic movements while learning to reach their goals, can lead to conflicts with human pedestrians due to their homogeneity. To tackle this problem, we introduce an efficient method that enhances agent diversity within a single policy by maximizing an information-theoretic objective. This diversity enriches each agent's experiences, improving its adaptability to unseen crowd behaviors. In assessing an agent's robustness against unseen crowds, we propose diverse scenarios inspired by pedestrian crowd behaviors. Our behavior-conditioned policies outperform existing works in these challenging scenes, reducing potential collisions without additional time or travel.