# Subequivariant Reinforcement Learning Framework for Coordinated Motion Control

arXiv: https://arxiv.org/abs/2403.15100 · HTML: https://mlsanity.com/paper/2403.15100/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 29 of its 52 references that are in the map.
- First posted on arXiv 22 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 29 of 29)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Exploration in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2205.00824/) (2022)
- [Graph Convolutional Reinforcement Learning](https://mlsanity.com/paper/1810.09202/) (2018)
- [Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots](https://mlsanity.com/paper/2103.14295/) (2021)
- [Explainability in Deep Reinforcement Learning](https://mlsanity.com/paper/2008.06693/) (2020)
- [Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networks](https://mlsanity.com/paper/2203.00199/) (2022)
- [Equivariant Graph Mechanics Networks with Constraints](https://mlsanity.com/paper/2203.06442/) (2022)
- [Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2006.11438/) (2020)
- [Learning to schedule job-shop problems: Representation and policy learning using graph neural network and reinforcement learning](https://mlsanity.com/paper/2106.01086/) (2021)
- [Reinforced Genetic Algorithm for Structure-based Drug Design](https://mlsanity.com/paper/2211.16508/) (2022)
- [Deep Reinforcement Learning in Computer Vision: A Comprehensive Survey](https://mlsanity.com/paper/2108.11510/) (2021)
- [Learning Physical Dynamics with Subequivariant Graph Neural Networks](https://mlsanity.com/paper/2210.06876/) (2022)
- [Low Dimensional Invariant Embeddings for Universal Geometric Learning](https://mlsanity.com/paper/2205.02956/) (2022)
- [JOLO-GCN: Mining Joint-Centered Light-Weight Information for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2011.07787/) (2020)
- [Graph Neural Network Reinforcement Learning for Autonomous Mobility-on-Demand Systems](https://mlsanity.com/paper/2104.11434/) (2021)
- [IronMan: GNN-assisted Design Space Exploration in High-Level Synthesis via Reinforcement Learning](https://mlsanity.com/paper/2102.08138/) (2021)
- [Joint-bone Fusion Graph Convolutional Network for Semi-supervised Skeleton Action Recognition](https://mlsanity.com/paper/2202.04075/) (2022)
- [Deep Multi-Agent Reinforcement Learning with Discrete-Continuous Hybrid Action Spaces](https://mlsanity.com/paper/1903.04959/) (2019)
- [Graph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments](https://mlsanity.com/paper/2006.12576/) (2020)
- [Comparing Popular Simulation Environments in the Scope of Robotics and Reinforcement Learning](https://mlsanity.com/paper/2103.04616/) (2021)
- [Finding Diverse and Predictable Subgraphs for Graph Domain Generalization](https://mlsanity.com/paper/2206.09345/) (2022)
- [Hybrid Zero Dynamics Inspired Feedback Control Policy Design for 3D Bipedal Locomotion using Reinforcement Learning](https://mlsanity.com/paper/1910.01748/) (2019)
- [Snowflake: Scaling GNNs to High-Dimensional Continuous Control via Parameter Freezing](https://mlsanity.com/paper/2103.01009/) (2021)
- [Graph Attention Network-based Multi-agent Reinforcement Learning for Slicing Resource Management in Dense Cellular Network](https://mlsanity.com/paper/2108.05063/) (2021)
- [Performing Deep Recurrent Double Q-Learning for Atari Games](https://mlsanity.com/paper/1908.06040/) (2019)

## Built on by (most cited first, 2 of 2)

- [POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfinding](https://mlsanity.com/paper/2407.14931/) (2024)
- [Towards Collaborative Intelligence: Propagating Intentions and Reasoning for Multi-Agent Coordination with Large Language Models](https://mlsanity.com/paper/2407.12532/) (2024)

## Abstract (arXiv)

Effective coordination is crucial for motion control with reinforcement learning, especially as the complexity of agents and their motions increases. However, many existing methods struggle to account for the intricate dependencies between joints. We introduce CoordiGraph, a novel architecture that leverages subequivariant principles from physics to enhance coordination of motion control with reinforcement learning. This method embeds the principles of equivariance as inherent patterns in the learning process under gravity influence, which aids in modeling the nuanced relationships between joints vital for motion control. Through extensive experimentation with sophisticated agents in diverse environments, we highlight the merits of our approach. Compared to current leading methods, CoordiGraph notably enhances generalization and sample efficiency.