# Efficient Morphology-Control Co-Design via Stackelberg Proximal Policy Optimization

arXiv: https://arxiv.org/abs/2603.15388 · HTML: https://mlsanity.com/paper/2603.15388/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 26 of its 50 references that are in the map.
- First posted on arXiv 16 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 26 of 26)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Embodied Intelligence via Learning and Evolution](https://mlsanity.com/paper/2102.02202/) (2021)
- [On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models](https://mlsanity.com/paper/1511.09249/) (2015)
- [Scalable Co-Optimization of Morphology and Control in Embodied Machines](https://mlsanity.com/paper/1706.06133/) (2017)
- [Sample-Efficient Learning of Stackelberg Equilibria in General-Sum Games](https://mlsanity.com/paper/2102.11494/) (2021)
- [Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design](https://mlsanity.com/paper/2110.03659/) (2021)
- [Neural Graph Evolution: Towards Efficient Automatic Robot Design](https://mlsanity.com/paper/1906.05370/) (2019)
- [Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation](https://mlsanity.com/paper/2205.03076/) (2022)
- [Universal Morphology Control via Contextual Modulation](https://mlsanity.com/paper/2302.11070/) (2023)
- [Decentralized Cooperative Reinforcement Learning with Hierarchical Information Structure](https://mlsanity.com/paper/2111.00781/) (2021)
- [Symmetry-Aware Robot Design with Structured Subgroups](https://mlsanity.com/paper/2306.00036/) (2023)
- [Oracles & Followers: Stackelberg Equilibria in Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2210.11942/) (2022)
- [Scalable sim-to-real transfer of soft robot designs](https://mlsanity.com/paper/1911.10290/) (2019)
- [BodyGen: Advancing Towards Efficient Embodiment Co-Design](https://mlsanity.com/paper/2503.00533/) (2025)
- [N-LIMB: Neural Limb Optimization for Efficient Morphological Design](https://mlsanity.com/paper/2207.11773/) (2022)
- [Reinforcement learning for freeform robot design](https://mlsanity.com/paper/2310.05670/) (2023)
- [A First Order Meta Stackelberg Method for Robust Federated Learning](https://mlsanity.com/paper/2306.13800/) (2023)
- [DittoGym: Learning to Control Soft Shape-Shifting Robots](https://mlsanity.com/paper/2401.13231/) (2024)
- [Model-free Reinforcement Learning for Stochastic Stackelberg Security Games](https://mlsanity.com/paper/2005.11853/) (2020)
- [Spatial-Temporal Moving Target Defense: A Markov Stackelberg Game Model](https://mlsanity.com/paper/2002.10390/) (2020)
- [Stackelberg Games for Learning Emergent Behaviors During Competitive Autocurricula](https://mlsanity.com/paper/2305.03735/) (2023)
- [CompetEvo: Towards Morphological Evolution from Competition](https://mlsanity.com/paper/2405.18300/) (2024)
- [Subequivariant Reinforcement Learning in 3D Multi-Entity Physical Environments](https://mlsanity.com/paper/2407.12505/) (2024)
- [Function Approximation for Solving Stackelberg Equilibrium in Large Perfect Information Games](https://mlsanity.com/paper/2212.14431/) (2022)

## Built on by (most cited first, 3 of 3)

- [Environment Parameter Gradient Theorem for Co-Design in Reinforcement Learning](https://mlsanity.com/paper/2607.12590/) (2026)
- [Bridging Body and Brain: Gene-Driven Morphology--Control Co-Design](https://mlsanity.com/paper/2609.31329/) (2026)
- [Co-design Gym: A Unified Benchmark for Embodiment-Policy Co-optimization](https://mlsanity.com/paper/2610.02366/) (2026)

## Abstract (arXiv)

Morphology-control co-design concerns the coupled optimization of an agent's body structure and control policy. This problem exhibits a bi-level structure, where the control dynamically adapts to the morphology to maximize performance. Existing methods typically neglect the control's adaptation dynamics by adopting a single-level formulation that treats the control policy as fixed when optimizing morphology. This can lead to inefficient optimization, as morphology updates may be misaligned with control adaptation. In this paper, we revisit the co-design problem from a game-theoretic perspective, modeling the intrinsic coupling between morphology and control as a novel variant of a Stackelberg game. We propose Stackelberg Proximal Policy Optimization (Stackelberg PPO), which explicitly incorporates the control's adaptation dynamics into morphology optimization. By modeling this intrinsic coupling, our method aligns morphology updates with control adaptation, thereby stabilizing training and improving learning efficiency. Experiments across diverse co-design tasks demonstrate that Stackelberg PPO outperforms standard PPO in both stability and final performance, opening the way for dramatically more efficient robotics designs.