# Reinforcement Learning for Adaptive Planner Parameter Tuning: A Perspective on Hierarchical Architecture

arXiv: https://arxiv.org/abs/2503.18366 · HTML: https://mlsanity.com/paper/2503.18366/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 24 Mar 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 16 of 16)

- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey](https://mlsanity.com/paper/2307.04370/) (2023)
- [Motion Planning and Control for Mobile Robot Navigation Using Machine Learning: a Survey](https://mlsanity.com/paper/2011.13112/) (2020)
- [Differentiable Gaussian Process Motion Planning](https://mlsanity.com/paper/1907.09591/) (2019)
- [Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectives](https://mlsanity.com/paper/2402.02968/) (2024)
- [APPLD: Adaptive Planner Parameter Learning from Demonstration](https://mlsanity.com/paper/2004.00116/) (2020)
- [APPL: Adaptive Planner Parameter Learning](https://mlsanity.com/paper/2105.07620/) (2021)
- [APPLI: Adaptive Planner Parameter Learning From Interventions](https://mlsanity.com/paper/2011.00400/) (2020)
- [Benchmarking Reinforcement Learning Techniques for Autonomous Navigation](https://mlsanity.com/paper/2210.04839/) (2022)
- [APPLR: Adaptive Planner Parameter Learning from Reinforcement](https://mlsanity.com/paper/2011.00397/) (2020)
- [APPLE: Adaptive Planner Parameter Learning from Evaluative Feedback](https://mlsanity.com/paper/2108.09801/) (2021)
- [A review of mobile robot motion planning methods: from classical motion planning workflows to reinforcement learning-based architectures](https://mlsanity.com/paper/2108.13619/) (2021)
- [ROS Navigation Tuning Guide](https://mlsanity.com/paper/1706.09068/) (2017)
- [From Agile Ground to Aerial Navigation: Learning from Learned Hallucination](https://mlsanity.com/paper/2108.09793/) (2021)
- [LiCS: Navigation using Learned-imitation on Cluttered Space](https://mlsanity.com/paper/2406.14947/) (2024)

## Built on by (most cited first, 1 of 1)

- [HiMAC: Hierarchical Macro-Micro Learning for Long-Horizon LLM Agents](https://mlsanity.com/paper/2603.00977/) (2026)

## Abstract (arXiv)

Automatic parameter tuning methods for planning algorithms, which integrate pipeline approaches with learning-based techniques, are regarded as promising due to their stability and capability to handle highly constrained environments. While existing parameter tuning methods have demonstrated considerable success, further performance improvements require a more structured approach. In this paper, we propose a hierarchical architecture for reinforcement learning-based parameter tuning. The architecture introduces a hierarchical structure with low-frequency parameter tuning, mid-frequency planning, and high-frequency control, enabling concurrent enhancement of both upper-layer parameter tuning and lower-layer control through iterative training. Experimental evaluations in both simulated and real-world environments show that our method surpasses existing parameter tuning approaches. Furthermore, our approach achieves first place in the Benchmark for Autonomous Robot Navigation (BARN) Challenge.