# DRL-VO: Learning to Navigate Through Crowded Dynamic Scenes Using Velocity Obstacles

arXiv: https://arxiv.org/abs/2301.06512 · HTML: https://mlsanity.com/paper/2301.06512/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2023.

## Built on by (most cited first, 30 of 37)

- [No Regrets: Investigating and Improving Regret Approximations for Curriculum Discovery](https://mlsanity.com/paper/2408.15099/) (2024)
- [SICNav-Diffusion: Safe and Interactive Crowd Navigation with Diffusion Trajectory Predictions](https://mlsanity.com/paper/2503.08858/) (2025)
- [HEIGHT: Heterogeneous Interaction Graph Transformer for Robot Navigation in Crowded and Constrained Environments](https://mlsanity.com/paper/2411.12150/) (2024)
- [SELFI: Autonomous Self-Improvement with Reinforcement Learning for Social Navigation](https://mlsanity.com/paper/2403.00991/) (2024)
- [PathRL: An End-to-End Path Generation Method for Collision Avoidance via Deep Reinforcement Learning](https://mlsanity.com/paper/2310.13295/) (2023)
- [Uncertainty-Aware DRL for Autonomous Vehicle Crowd Navigation in Shared Space](https://mlsanity.com/paper/2405.13969/) (2024)
- [Towards Generalizable Safety in Crowd Navigation via Conformal Uncertainty Handling](https://mlsanity.com/paper/2508.05634/) (2025)
- [BEVNav: Robot Autonomous Navigation Via Spatial-Temporal Contrastive Learning in Bird's-Eye View](https://mlsanity.com/paper/2409.01646/) (2024)
- [Disentangling Uncertainty for Safe Social Navigation using Deep Reinforcement Learning](https://mlsanity.com/paper/2409.10655/) (2024)
- [Crowd-FM: Learned Optimal Selection of Conditional Flow Matching-generated Trajectories for Crowd Navigation](https://mlsanity.com/paper/2602.06698/) (2026)
- [Arena-Web -- A Web-based Development and Benchmarking Platform for Autonomous Navigation Approaches](https://mlsanity.com/paper/2302.02898/) (2023)
- [Enhancing Deep Reinforcement Learning-based Robot Navigation Generalization through Scenario Augmentation](https://mlsanity.com/paper/2503.01146/) (2025)
- [EmoBipedNav: Emotion-aware Social Navigation for Bipedal Robots with Deep Reinforcement Learning](https://mlsanity.com/paper/2503.12538/) (2025)
- [Reinforcement Learning with Data Bootstrapping for Dynamic Subgoal Pursuit in Humanoid Robot Navigation](https://mlsanity.com/paper/2506.02206/) (2025)
- [HABIT: Human Action Benchmark for Interactive Traffic in CARLA](https://mlsanity.com/paper/2511.19109/) (2025)
- [TRANS: Terrain-aware Reinforcement Learning for Agile Navigation of Quadruped Robots under Social Interactions](https://mlsanity.com/paper/2602.12724/) (2026)
- [Don't Freeze, Don't Crash: Extending the Safe Operating Range of Neural Navigation in Dense Crowds](https://mlsanity.com/paper/2603.06729/) (2026)
- [NavRL++: A System-Level Framework for Improving Sim-to-Real Transfer in Reinforcement Learning-Based Robot Navigation](https://mlsanity.com/paper/2605.15559/) (2026)
- [RUMOR: Reinforcement learning for Understanding a Model of the Real World for Navigation in Dynamic Environments](https://mlsanity.com/paper/2404.16672/) (2024)
- [Structured Graph Network for Constrained Robot Crowd Navigation with Low Fidelity Simulation](https://mlsanity.com/paper/2405.16830/) (2024)
- [ColorDynamic: Generalizable, Scalable, Real-time, End-to-end Local Planner for Unstructured and Dynamic Environments](https://mlsanity.com/paper/2502.19892/) (2025)
- [Beyond Visibility Limits: A DRL-Based Navigation Strategy for Unexpected Obstacles](https://mlsanity.com/paper/2503.01127/) (2025)
- [NavG: Risk-Aware Navigation in Crowded Environments Based on Reinforcement Learning with Guidance Points](https://mlsanity.com/paper/2503.02111/) (2025)
- [AOLO: Analysis and Optimization For Low-Carbon Oriented Wireless Large Language Model Services](https://mlsanity.com/paper/2503.04418/) (2025)
- [AttentionSwarm: Reinforcement Learning with Attention Control Barier Function for Crazyflie Drones in Dynamic Environments](https://mlsanity.com/paper/2503.07376/) (2025)
- [MAER-Nav: Bidirectional Motion Learning Through Mirror-Augmented Experience Replay for Robot Navigation](https://mlsanity.com/paper/2503.23908/) (2025)
- [DRL-TH: Jointly Utilizing Temporal Graph Attention and Hierarchical Fusion for UGV Navigation in Crowded Environments](https://mlsanity.com/paper/2512.24284/) (2025)
- [Sim2Sea: Sim-to-Real Policy Transfer for Maritime Vessel Navigation in Congested Waters](https://mlsanity.com/paper/2603.04057/) (2026)
- [GIANT - Global Path Integration and Attentive Graph Networks for Multi-Agent Trajectory Planning](https://mlsanity.com/paper/2603.04659/) (2026)
- [Beyond Specialization: Robust Reinforcement Learning Navigation via Procedural Map Generators](https://mlsanity.com/paper/2605.02528/) (2026)

## Abstract (arXiv)

This paper proposes a novel learning-based control policy with strong generalizability to new environments that enables a mobile robot to navigate autonomously through spaces filled with both static obstacles and dense crowds of pedestrians. The policy uses a unique combination of input data to generate the desired steering angle and forward velocity: a short history of lidar data, kinematic data about nearby pedestrians, and a sub-goal point. The policy is trained in a reinforcement learning setting using a reward function that contains a novel term based on velocity obstacles to guide the robot to actively avoid pedestrians and move towards the goal. Through a series of 3D simulated experiments with up to 55 pedestrians, this control policy is able to achieve a better balance between collision avoidance and speed (i.e., higher success rate and faster average speed) than state-of-the-art model-based and learning-based policies, and it also generalizes better to different crowd sizes and unseen environments. An extensive series of hardware experiments demonstrate the ability of this policy to directly work in different real-world environments with different crowd sizes with zero retraining. Furthermore, a series of simulated and hardware experiments show that the control policy also works in highly constrained static environments on a different robot platform without any additional training. Lastly, several important lessons that can be applied to other robot learning systems are summarized. Multimedia demonstrations are available at https://www.youtube.com/watch?v=KneELRT8GzU&list=PLouWbAcP4zIvPgaARrV223lf2eiSR-eSS.