# ReProHRL: Towards Multi-Goal Navigation in the Real World using Hierarchical Agents

arXiv: https://arxiv.org/abs/2308.08737 · HTML: https://mlsanity.com/paper/2308.08737/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 17 Aug 2023.

## Builds on (most cited first, 18 of 18)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05143/) (2016)
- [Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learning](https://mlsanity.com/paper/1812.00971/) (2018)
- [Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning](https://mlsanity.com/paper/1712.07294/) (2017)
- [Sim2real transfer learning for 3D human pose estimation: motion to the rescue](https://mlsanity.com/paper/1907.02499/) (2019)
- [TACTO: A Fast, Flexible, and Open-source Simulator for High-Resolution Vision-based Tactile Sensors](https://mlsanity.com/paper/2012.08456/) (2020)
- [HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulators](https://mlsanity.com/paper/1910.11432/) (2019)
- [Bi-directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation Agents](https://mlsanity.com/paper/2011.12421/) (2020)
- [Motion Perception in Reinforcement Learning with Dynamic Objects](https://mlsanity.com/paper/1901.03162/) (2019)
- [Air Learning: A Deep Reinforcement Learning Gym for Autonomous Aerial Robot Visual Navigation](https://mlsanity.com/paper/1906.00421/) (2019)
- [Sniffy Bug: A Fully Autonomous Swarm of Gas-Seeking Nano Quadcopters in Cluttered Environments](https://mlsanity.com/paper/2107.05490/) (2021)
- [YOLO-Z: Improving small object detection in YOLOv5 for autonomous vehicles](https://mlsanity.com/paper/2112.11798/) (2021)
- [Sim2real for Autonomous Vehicle Control using Executable Digital Twin](https://mlsanity.com/paper/2208.06226/) (2022)
- [Interactive Hierarchical Guidance using Language](https://mlsanity.com/paper/2110.04649/) (2021)
- [On the use of Deep Autoencoders for Efficient Embedded Reinforcement Learning](https://mlsanity.com/paper/1903.10404/) (2019)
- [Towards an Interpretable Hierarchical Agent Framework using Semantic Goals](https://mlsanity.com/paper/2210.08412/) (2022)
- [Local Navigation and Docking of an Autonomous Robot Mower using Reinforcement Learning and Computer Vision](https://mlsanity.com/paper/2101.06248/) (2021)

## Built on by (most cited first, 5 of 5)

- [Squeezed Edge YOLO: Onboard Object Detection on Edge Devices](https://mlsanity.com/paper/2312.11716/) (2023)
- [TinyVQA: Compact Multimodal Deep Neural Network for Visual Question Answering on Resource-Constrained Devices](https://mlsanity.com/paper/2404.03574/) (2024)
- [MambaLiteSR: Image Super-Resolution with Low-Rank Mamba using Knowledge Distillation](https://mlsanity.com/paper/2502.14090/) (2025)
- [EdgeNavMamba: Mamba Optimized Object Detection for Energy Efficient Edge Devices](https://mlsanity.com/paper/2510.14946/) (2025)
- [Energy-Aware FPGA Implementation of Spiking Neural Network with LIF Neurons](https://mlsanity.com/paper/2411.01628/) (2024)

## Abstract (arXiv)

Robots have been successfully used to perform tasks with high precision. In real-world environments with sparse rewards and multiple goals, learning is still a major challenge and Reinforcement Learning (RL) algorithms fail to learn good policies. Training in simulation environments and then fine-tuning in the real world is a common approach. However, adapting to the real-world setting is a challenge. In this paper, we present a method named Ready for Production Hierarchical RL (ReProHRL) that divides tasks with hierarchical multi-goal navigation guided by reinforcement learning. We also use object detectors as a pre-processing step to learn multi-goal navigation and transfer it to the real world. Empirical results show that the proposed ReProHRL method outperforms the state-of-the-art baseline in simulation and real-world environments in terms of both training time and performance. Although both methods achieve a 100% success rate in a simple environment for single goal-based navigation, in a more complex environment and multi-goal setting, the proposed method outperforms the baseline by 18% and 5%, respectively. For the real-world implementation and proof of concept demonstration, we deploy the proposed method on a nano-drone named Crazyflie with a front camera to perform multi-goal navigation experiments.