# COMPOSER: Scalable and Robust Modular Policies for Snake Robots

arXiv: https://arxiv.org/abs/2310.00871 · HTML: https://mlsanity.com/paper/2310.00871/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 42 references that are in the map.
- First posted on arXiv 2 Oct 2023.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.08839/) (2020)
- [Learning Multiagent Communication with Backpropagation](https://mlsanity.com/paper/1605.07736/) (2016)
- [Trust Region Policy Optimisation in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2109.11251/) (2021)
- [Multi-Agent Reinforcement Learning is a Sequence Modeling Problem](https://mlsanity.com/paper/2205.14953/) (2022)
- [Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks](https://mlsanity.com/paper/1812.09755/) (2018)
- [One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control](https://mlsanity.com/paper/2007.04976/) (2020)
- [Hardware Conditioned Policies for Multi-Robot Transfer Learning](https://mlsanity.com/paper/1811.09864/) (2018)
- [Learning Modular Robot Control Policies](https://mlsanity.com/paper/2105.10049/) (2021)
- [Evolving Modular Soft Robots without Explicit Inter-Module Communication using Local Self-Attention](https://mlsanity.com/paper/2204.06481/) (2022)
- [Distributed Learning of Decentralized Control Policies for Articulated Mobile Robots](https://mlsanity.com/paper/1901.08537/) (2019)
- [Directional Compliance in Obstacle-Aided Navigation for Snake Robots](https://mlsanity.com/paper/2003.01774/) (2020)
- [Mechanical Intelligence Simplifies Control in Terrestrial Limbless Locomotion](https://mlsanity.com/paper/2304.08652/) (2023)
- [Robotic modeling of snake traversing large, smooth obstacles reveals stability benefits of body compliance](https://mlsanity.com/paper/2002.09711/) (2020)
- [Reconfigurable Robot Control Using Flexible Coupling Mechanisms](https://mlsanity.com/paper/2306.16265/) (2023)

## Built on by (most cited first, 5 of 5)

- [Learning Multi-Agent Loco-Manipulation for Long-Horizon Quadrupedal Pushing](https://mlsanity.com/paper/2411.07104/) (2024)
- [Multi-agent Coordination via Flow Matching](https://mlsanity.com/paper/2511.05005/) (2025)
- [MARLIN: Multi-Agent Reinforcement Learning Guided by Language-Based Inter-Robot Negotiation](https://mlsanity.com/paper/2410.14383/) (2024)
- [Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges](https://mlsanity.com/paper/2512.04770/) (2025)
- [A Quantitative Comparison of Centralised and Distributed Reinforcement Learning-Based Control for Soft Robotic Arms](https://mlsanity.com/paper/2511.02192/) (2025)

## Abstract (arXiv)

Snake robots have showcased remarkable compliance and adaptability in their interaction with environments, mirroring the traits of their natural counterparts. While their hyper-redundant and high-dimensional characteristics add to this adaptability, they also pose great challenges to robot control. Instead of perceiving the hyper-redundancy and flexibility of snake robots as mere challenges, there lies an unexplored potential in leveraging these traits to enhance robustness and generalizability at the control policy level. We seek to develop a control policy that effectively breaks down the high dimensionality of snake robots while harnessing their redundancy. In this work, we consider the snake robot as a modular robot and formulate the control of the snake robot as a cooperative Multi-Agent Reinforcement Learning (MARL) problem. Each segment of the snake robot functions as an individual agent. Specifically, we incorporate a self-attention mechanism to enhance the cooperative behavior between agents. A high-level imagination policy is proposed to provide additional rewards to guide the low-level control policy. We validate the proposed method COMPOSER with five snake robot tasks, including goal reaching, wall climbing, shape formation, tube crossing, and block pushing. COMPOSER achieves the highest success rate across all tasks when compared to a centralized baseline and four modular policy baselines. Additionally, we show enhanced robustness against module corruption and significantly superior zero-shot generalizability in our proposed method. The videos of this work are available on our project page: https://sites.google.com/view/composer-snake/.