# RUMOR: Reinforcement learning for Understanding a Model of the Real World for Navigation in Dynamic Environments

arXiv: https://arxiv.org/abs/2404.16672 · HTML: https://mlsanity.com/paper/2404.16672/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 35 of its 64 references that are in the map.
- First posted on arXiv 25 Apr 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 30 of 35)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://mlsanity.com/paper/1710.02298/) (2017)
- [Distributed Prioritized Experience Replay](https://mlsanity.com/paper/1803.00933/) (2018)
- [Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots](https://mlsanity.com/paper/2310.13724/) (2023)
- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learning](https://mlsanity.com/paper/1809.08835/) (2018)
- [Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.07845/) (2016)
- [Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1805.01956/) (2018)
- [Core Challenges of Social Robot Navigation: A Survey](https://mlsanity.com/paper/2103.05668/) (2021)
- [Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning](https://mlsanity.com/paper/1709.10082/) (2017)
- [Principles and Guidelines for Evaluating Social Robot Navigation Algorithms](https://mlsanity.com/paper/2306.16740/) (2023)
- [Intention Aware Robot Crowd Navigation with Attention-Based Interaction Graph](https://mlsanity.com/paper/2203.01821/) (2022)
- [Collision Avoidance in Pedestrian-Rich Environments with Deep Reinforcement Learning](https://mlsanity.com/paper/1910.11689/) (2019)
- [Relational Graph Learning for Crowd Navigation](https://mlsanity.com/paper/1909.13165/) (2019)
- [A Survey on Deep Domain Adaptation for LiDAR Perception](https://mlsanity.com/paper/2106.02377/) (2021)
- [From Perception to Decision: A Data-driven Approach to End-to-end Motion Planning for Autonomous Ground Robots](https://mlsanity.com/paper/1609.07910/) (2016)
- [A Survey on Socially Aware Robot Navigation: Taxonomy and Future Challenges](https://mlsanity.com/paper/2311.06922/) (2023)
- [DRL-VO: Learning to Navigate Through Crowded Dynamic Scenes Using Velocity Obstacles](https://mlsanity.com/paper/2301.06512/) (2023)
- [DenseCAvoid: Real-time Navigation in Dense Crowds using Anticipatory Behaviors](https://mlsanity.com/paper/2002.03038/) (2020)
- [Learning Perception-Aware Agile Flight in Cluttered Environments](https://mlsanity.com/paper/2210.01841/) (2022)
- [Model Predictive Contouring Control for Collision Avoidance in Unstructured Dynamic Environments](https://mlsanity.com/paper/2010.10190/) (2020)
- [Dynamically Feasible Deep Reinforcement Learning Policy for Robot Navigation in Dense Mobile Crowds](https://mlsanity.com/paper/2010.14838/) (2020)
- [Arena-Rosnav: Towards Deployment of Deep-Reinforcement-Learning-Based Obstacle Avoidance into Conventional Autonomous Navigation Systems](https://mlsanity.com/paper/2104.03616/) (2021)
- [Rethinking Social Robot Navigation: Leveraging the Best of Two Worlds](https://mlsanity.com/paper/2309.13466/) (2023)
- [From Crowd Motion Prediction to Robot Navigation in Crowds](https://mlsanity.com/paper/2303.01424/) (2023)
- [Crowd-Aware Robot Navigation for Pedestrians with Multiple Collision Avoidance Strategies via Map-based Deep Reinforcement Learning](https://mlsanity.com/paper/2109.02541/) (2021)
- [Winding Through: Crowd Navigation via Topological Invariance](https://mlsanity.com/paper/2109.05084/) (2021)
- [Arena-Bench: A Benchmarking Suite for Obstacle Avoidance Approaches in Highly Dynamic Environments](https://mlsanity.com/paper/2206.05728/) (2022)
- [NavDreams: Towards Camera-Only RL Navigation Among Humans](https://mlsanity.com/paper/2203.12299/) (2022)
- [Learning to Socially Navigate in Pedestrian-rich Environments with Interaction Capacity](https://mlsanity.com/paper/2203.16154/) (2022)

## Abstract (arXiv)

Autonomous navigation in dynamic environments is a complex but essential task for autonomous robots, with recent deep reinforcement learning approaches showing promising results. However, the complexity of the real world makes it infeasible to train agents in every possible scenario configuration. Moreover, existing methods typically overlook factors such as robot kinodynamic constraints, or assume perfect knowledge of the environment. In this work, we present RUMOR, a novel planner for differential-drive robots that uses deep reinforcement learning to navigate in highly dynamic environments. Unlike other end-to-end DRL planners, it uses a descriptive robocentric velocity space model to extract the dynamic environment information, enhancing training effectiveness and scenario interpretation. Additionally, we propose an action space that inherently considers robot kinodynamics and train it in a simulator that reproduces the real world problematic aspects, reducing the gap between the reality and simulation. We extensively compare RUMOR with other state-of-the-art approaches, demonstrating a better performance, and provide a detailed analysis of the results. Finally, we validate RUMOR's performance in real-world settings by deploying it on a ground robot. Our experiments, conducted in crowded scenarios and unseen environments, confirm the algorithm's robustness and transferability.