High-Precision Formation Control for Heterogeneous Multi-Robot Systems via Hierarchical Hybrid Physics-Informed Deep Reinforcement Learning
On the map: Deep reinforcement learning methods → RL for robot navigation and UAV control
Abstract (arXiv)
Existing classical control methods commonly require precise models and struggle to cope with model uncertainties and external disturbances, while end-to-end reinforcement learning (RL) approaches suffer from low sample efficiency and poor convergence. To overcome these challenges, this paper proposes a hierarchical hybrid physics-informed deep reinforcement learning (HHy-PIDRL) framework, aiming to realize high-precision, highly responsive formation control for heterogeneous multi-robot systems (HMRSs). The proposed framework contains two layers. Specifically, first, the upper layer designs an autonomous navigation policy network for Ackermann-steering leader based on the Soft Actor-Critic (SAC) deep reinforcement learning (DRL) algorithm. Second, the lower module integrates a high-fidelity physical feed-forward controller, a classical proportional-derivative (PD) controller, and an adaptive DRL residual controller to propose an effective hybrid model and DRL (HM-DRL)-based formation control policy network. Third, a unique hierarchical reward function is designed for training Omnidirectional followers, which effectively guides agents toward a refined, stable control policy. Experimental results demonstrate that, the success rate of both the upper-layer autonomous navigation policy network and the HM-DRL based formation control policy networks reach 100%. Meanwhile, ablation experiments are conducted to verify the validity and credibility of the proposed method.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. Builds on 7 of its 43 references that are in the map. First posted on arXiv 3 Jul 2026.
Builds on
7 of 43 references.
References · 7
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2020BADGR: An Autonomous Self-Supervised Learning-Based Navigation Systemcited by 102
- 2019Online Trajectory Generation with Distributed Model Predictive Control for Multi-Robot Motion Planningcited by 14
- 2023Safe Multi-Agent Reinforcement Learning for Behavior-Based Cooperative Navigationcited by 6
- 2024Graph Neural Network-based Multi-agent Reinforcement Learning for Resilient Distributed Coordination of Multi-Robot Systemscited by 5
- 2023CDT-Dijkstra: Fast Planning of Globally Optimal Paths for All Points in 2D Continuous Spacecited by 1
- 2025Scalable Safe Multi-Agent Reinforcement Learning for Multi-Agent Systemcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Taxonomy and Trends in Reinforcement Learning for Robotics and Control Systems: A Structured Review
- 2024Deep Reinforcement Learning for Decentralized Multi-Robot Control: A DQN Approach to Robustness and Information Integration
- 2025A Novel Multi-Timescale Stability-Preserving Hierarchical Reinforcement Learning Controller Framework for Adaptive Control in High-Dimensional Dynamical Systems
- 2025Hybrid Motion Planning with Deep Reinforcement Learning for Mobile Robot Navigation
- 2025Multi-fidelity Reinforcement Learning Control for Complex Dynamical Systems
- 2025Re4MPC: Reactive Nonlinear MPC for Multi-model Motion Planning via Deep Reinforcement Learning
- 2024Integrating DeepRL with Robust Low-Level Control in Robotic Manipulators for Non-Repetitive Reaching Tasks
- 2026Physics-Informed Policy Optimization via Analytic Dynamics Regularization