# Multi-Modal Decentralized Reinforcement Learning for Modular Reconfigurable Lunar Robots

arXiv: https://arxiv.org/abs/2510.20347 · HTML: https://mlsanity.com/paper/2510.20347/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 23 Oct 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Offline Decentralized Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2108.01832/) (2021)
- [Learning Modular Robot Control Policies](https://mlsanity.com/paper/2105.10049/) (2021)
- [Reinforcement Learned Distributed Multi-Robot Navigation with Reciprocal Velocity Obstacle Shaped Rewards](https://mlsanity.com/paper/2203.10229/) (2022)
- [Distributed Reinforcement Learning for Robot Teams: A Review](https://mlsanity.com/paper/2204.03516/) (2022)
- [Distributed Learning of Decentralized Control Policies for Articulated Mobile Robots](https://mlsanity.com/paper/1901.08537/) (2019)
- [On Decentralizing Federated Reinforcement Learning in Multi-Robot Scenarios](https://mlsanity.com/paper/2207.09372/) (2022)
- [Robust and Modular Multi-Limb Synchronization in Motion Stack for Space Robots with Trajectory Clamping via Hypersphere](https://mlsanity.com/paper/2507.03934/) (2025)
- [Integration of Vision-based Object Detection and Grasping for Articulated Manipulator in Lunar Conditions](https://mlsanity.com/paper/2309.01055/) (2023)
- [Enhancing Autonomous Manipulator Control with Human-in-loop for Uncertain Assembly Environments](https://mlsanity.com/paper/2507.11006/) (2025)

## Built on by (most cited first, 1 of 1)

- [Distributed Multi Robot Lunar Cargo Transportation via Phase Decomposed Reinforcement Learning](https://mlsanity.com/paper/2607.00160/) (2026)

## Abstract (arXiv)

Modular reconfigurable robots suit task-specific space operations, but the combinatorial growth of morphologies hinders unified control. We propose a decentralized reinforcement learning (Dec-RL) scheme where each module learns its own policy: wheel modules use Soft Actor-Critic (SAC) for locomotion and 7-DoF limbs use Proximal Policy Optimization (PPO) for steering and manipulation, enabling zero-shot generalization to unseen configurations. In simulation, the steering policy achieved a mean absolute error of 3.63deg between desired and induced angles; the manipulation policy plateaued at 84.6 % success on a target-offset criterion; and the wheel policy cut average motor torque by 95.4 % relative to baseline while maintaining 99.6 % success. Lunar-analogue field tests validated zero-shot integration for autonomous locomotion, steering, and preliminary alignment for reconfiguration. The system transitioned smoothly among synchronous, parallel, and sequential modes for Policy Execution, without idle states or control conflicts, indicating a scalable, reusable, and robust approach for modular lunar robots.