Heterogeneous Multi-Expert Reinforcement Learning for Long-Horizon Multi-Goal Tasks in Autonomous Forklifts
On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation
Abstract (arXiv)
Autonomous mobile manipulation in unstructured warehouses requires a balance between efficient large-scale navigation and high-precision object interaction. Traditional end-to-end learning approaches often struggle to handle the conflicting demands of these distinct phases. Navigation relies on robust decision-making over large spaces, while manipulation needs high sensitivity to fine local details. Forcing a single network to learn these different objectives simultaneously often causes optimization interference, where improving one task degrades the other. To address these limitations, we propose a Heterogeneous Multi-Expert Reinforcement Learning (HMER) framework tailored for autonomous forklifts. HMER decomposes long-horizon tasks into specialized sub-policies controlled by a Semantic Task Planner. This structure separates macro-level navigation from micro-level manipulation, allowing each expert to focus on its specific action space without interference. The planner coordinates the sequential execution of these experts, bridging the gap between task planning and continuous control. Furthermore, to solve the problem of sparse exploration, we introduce a Hybrid Imitation-Reinforcement Training Strategy. This method uses expert demonstrations to initialize the policy and Reinforcement Learning for fine-tuning. Experiments in Gazebo simulations show that HMER significantly outperforms sequential and end-to-end baselines. Our method achieves a task success rate of 94.2% (compared to 62.5% for baselines), reduces operation time by 21.4%, and maintains placement error within 1.5 cm, validating its efficacy for precise material handling.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 22 references that are in the map. First posted on arXiv 12 Jan 2026.
Builds on
12 of 22 references.
References · 12
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2016Generative Adversarial Imitation Learningcited by 1,597
- 2015End-to-End Training of Deep Visuomotor Policiescited by 875
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
- 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
- 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
- 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
- 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
- 2020Integrated Task and Motion Planningcited by 290
- 2018Residual Reinforcement Learning for Robot Controlcited by 274
- 2021Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulationcited by 209
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ExpertGen: Scalable Sim-to-Real Expert Policy Learning from Imperfect Behavior Priors
- 2026Contact-Guided Exploration for Non-Prehensile Locomanipulation with Multi-Critic RL ↗
- 2026Abstracting Robot Manipulation Skills via Mixture-of-Experts Diffusion Policies
- 2026MobiAgent: Dual-Loop Recursive Policy Self-Improvement for Long-Horizon Mobile Manipulation
- 2024Learning Manipulation Tasks in Dynamic and Shared 3D Spaces
- 2024Guided Reinforcement Learning for Robust Multi-Contact Loco-Manipulation
- 2024SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation
- 2023Offline Skill Generalization via Task and Motion Planning