# Benchmarking Reinforcement Learning Techniques for Autonomous Navigation

arXiv: https://arxiv.org/abs/2210.04839 · HTML: https://mlsanity.com/paper/2210.04839/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 46 references that are in the map.
- First posted on arXiv 10 Oct 2022.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018)
- [Leveraging Procedural Generation to Benchmark Reinforcement Learning](https://mlsanity.com/paper/1912.01588/) (2019)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05143/) (2016)
- [Quantifying Generalization in Reinforcement Learning](https://mlsanity.com/paper/1812.02341/) (2018)
- [Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning](https://mlsanity.com/paper/1708.02596/) (2017)
- [Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design](https://mlsanity.com/paper/2012.02096/) (2020)
- [Lyapunov-based Safe Policy Optimization for Continuous Control](https://mlsanity.com/paper/1901.10031/) (2019)
- [Interactive Gibson Benchmark (iGibson 0.5): A Benchmark for Interactive Navigation in Cluttered Environments](https://mlsanity.com/paper/1910.14442/) (2019)
- [Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation](https://mlsanity.com/paper/1806.10729/) (2018)
- [Learning Navigation Behaviors End-to-End with AutoRL](https://mlsanity.com/paper/1809.10124/) (2018)
- [Safe Reinforcement Learning by Imagining the Near Future](https://mlsanity.com/paper/2202.07789/) (2022)
- [Motion Planning and Control for Mobile Robot Navigation Using Machine Learning: a Survey](https://mlsanity.com/paper/2011.13112/) (2020)
- [APPLD: Adaptive Planner Parameter Learning from Demonstration](https://mlsanity.com/paper/2004.00116/) (2020)
- [Benchmarking Metric Ground Navigation](https://mlsanity.com/paper/2008.13315/) (2020)
- [APPL: Adaptive Planner Parameter Learning](https://mlsanity.com/paper/2105.07620/) (2021)
- [APPLI: Adaptive Planner Parameter Learning From Interventions](https://mlsanity.com/paper/2011.00400/) (2020)
- [APPLR: Adaptive Planner Parameter Learning from Reinforcement](https://mlsanity.com/paper/2011.00397/) (2020)
- [APPLE: Adaptive Planner Parameter Learning from Evaluative Feedback](https://mlsanity.com/paper/2108.09801/) (2021)
- [Long Range Neural Navigation Policies for the Real World](https://mlsanity.com/paper/1903.09870/) (2019)
- [Autonomous Ground Navigation in Highly Constrained Spaces: Lessons learned from The BARN Challenge at ICRA 2022](https://mlsanity.com/paper/2208.10473/) (2022)

## Built on by (most cited first, 19 of 19)

- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [No Regrets: Investigating and Improving Regret Approximations for Curriculum Discovery](https://mlsanity.com/paper/2408.15099/) (2024)
- [DTG : Diffusion-based Trajectory Generation for Mapless Global Navigation](https://mlsanity.com/paper/2403.09900/) (2024)
- [Reinforcement Learning for Wheeled Mobility on Vertically Challenging Terrain](https://mlsanity.com/paper/2409.02383/) (2024)
- [Grounded Curriculum Learning](https://mlsanity.com/paper/2409.19816/) (2024)
- [A Dynamic Safety Shield for Safe and Efficient Reinforcement Learning of Navigation Tasks](https://mlsanity.com/paper/2412.04153/) (2024)
- [A Study on Learning Social Robot Navigation with Multimodal Perception](https://mlsanity.com/paper/2309.12568/) (2023)
- [Reward Training Wheels: Adaptive Auxiliary Rewards for Robotics Reinforcement Learning](https://mlsanity.com/paper/2503.15724/) (2025)
- [Optimistic critics can empower small actors](https://mlsanity.com/paper/2506.01016/) (2025)
- [RoboRAN: A Unified Robotics Framework for Reinforcement Learning-Based Autonomous Navigation](https://mlsanity.com/paper/2505.14526/) (2025)
- [GACL: Grounded Adaptive Curriculum Learning with Active Task and Performance Monitoring](https://mlsanity.com/paper/2508.02988/) (2025)
- [Vision-driven UAV River Following: Benchmarking with Safe Reinforcement Learning](https://mlsanity.com/paper/2409.08511/) (2024)
- [GUIDEd Agents: Enhancing Navigation Policies through Task-Specific Uncertainty Abstraction in Localization-Limited Environments](https://mlsanity.com/paper/2410.15178/) (2024)
- [Enhancing Robot Navigation Policies with Task-Specific Uncertainty Managements](https://mlsanity.com/paper/2505.13837/) (2025)
- [Dyna-LfLH: Learning Agile Navigation in Dynamic Environments from Learned Hallucination](https://mlsanity.com/paper/2403.17231/) (2024)
- [Reinforcement Learning for Adaptive Planner Parameter Tuning: A Perspective on Hierarchical Architecture](https://mlsanity.com/paper/2503.18366/) (2025)
- [SpikeStereoNet: A Brain-Inspired Framework for Stereo Depth Estimation from Spike Streams](https://mlsanity.com/paper/2505.19487/) (2025)
- [CARoL: Context-aware Adaptation for Robot Learning](https://mlsanity.com/paper/2506.07006/) (2025)
- [Adapting Reinforcement Learning for Path Planning in Constrained Parking Scenarios](https://mlsanity.com/paper/2601.22545/) (2026)

## Abstract (arXiv)

Deep reinforcement learning (RL) has brought many successes for autonomous robot navigation. However, there still exists important limitations that prevent real-world use of RL-based navigation systems. For example, most learning approaches lack safety guarantees; and learned navigation systems may not generalize well to unseen environments. Despite a variety of recent learning techniques to tackle these challenges in general, a lack of an open-source benchmark and reproducible learning methods specifically for autonomous navigation makes it difficult for roboticists to choose what learning methods to use for their mobile robots and for learning researchers to identify current shortcomings of general learning methods for autonomous navigation. In this paper, we identify four major desiderata of applying deep RL approaches for autonomous navigation: (D1) reasoning under uncertainty, (D2) safety, (D3) learning from limited trial-and-error data, and (D4) generalization to diverse and novel environments. Then, we explore four major classes of learning techniques with the purpose of achieving one or more of the four desiderata: memory-based neural network architectures (D1), safe RL (D2), model-based RL (D2, D3), and domain randomization (D4). By deploying these learning techniques in a new open-source large-scale navigation benchmark and real-world environments, we perform a comprehensive study aimed at establishing to what extent can these techniques achieve these desiderata for RL-based navigation systems.