# DittoGym: Learning to Control Soft Shape-Shifting Robots

arXiv: https://arxiv.org/abs/2401.13231 · HTML: https://mlsanity.com/paper/2401.13231/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 44 references that are in the map.
- First posted on arXiv 24 Jan 2024.

## Builds on (most cited first, 20 of 20)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning](https://mlsanity.com/paper/1611.04717/) (2016)
- [Embodied Intelligence via Learning and Evolution](https://mlsanity.com/paper/2102.02202/) (2021)
- [PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics](https://mlsanity.com/paper/2104.03311/) (2021)
- [ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics](https://mlsanity.com/paper/1810.01054/) (2018)
- [MetaMorph: Learning Universal Controllers with Transformers](https://mlsanity.com/paper/2203.11931/) (2022)
- [Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularity](https://mlsanity.com/paper/1902.05546/) (2019)
- [Reinforcement Learning for Improving Agent Design](https://mlsanity.com/paper/1810.03779/) (2018)
- [Discrete Sequential Prediction of Continuous Actions for Deep RL](https://mlsanity.com/paper/1705.05035/) (2017)
- [Evolution Gym: A Large-Scale Benchmark for Evolving Soft Robots](https://mlsanity.com/paper/2201.09863/) (2022)
- [Gradientless Descent: High-Dimensional Zeroth-Order Optimization](https://mlsanity.com/paper/1911.06317/) (2019)
- [Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design](https://mlsanity.com/paper/2110.03659/) (2021)
- [DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models](https://mlsanity.com/paper/2311.17053/) (2023)
- [Fit2Form: 3D Generative Model for Robot Gripper Form Design](https://mlsanity.com/paper/2011.06498/) (2020)
- [SoftZoo: A Soft Robot Co-design Benchmark For Locomotion In Diverse Environments](https://mlsanity.com/paper/2303.09555/) (2023)
- [GLSO: Grammar-guided Latent Space Optimization for Sample-efficient Robot Design Automation](https://mlsanity.com/paper/2209.11748/) (2022)

## Built on by (most cited first, 6 of 6)

- [MoE-DP: An MoE-Enhanced Diffusion Policy for Robust Long-Horizon Robotic Manipulation with Skill Decomposition and Failure Recovery](https://mlsanity.com/paper/2511.05007/) (2025)
- [BodyGen: Advancing Towards Efficient Embodiment Co-Design](https://mlsanity.com/paper/2503.00533/) (2025)
- [Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systems](https://mlsanity.com/paper/2507.14850/) (2025)
- [Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges](https://mlsanity.com/paper/2512.04770/) (2025)
- [Efficient Morphology-Control Co-Design via Stackelberg Proximal Policy Optimization](https://mlsanity.com/paper/2603.15388/) (2026)
- [IMBench: A Benchmark for Intuitive Robotic Manipulation](https://mlsanity.com/paper/2607.15641/) (2026)

## Abstract (arXiv)

Robot co-design, where the morphology of a robot is optimized jointly with a learned policy to solve a specific task, is an emerging area of research. It holds particular promise for soft robots, which are amenable to novel manufacturing techniques that can realize learned morphologies and actuators. Inspired by nature and recent novel robot designs, we propose to go a step further and explore the novel reconfigurable robots, defined as robots that can change their morphology within their lifetime. We formalize control of reconfigurable soft robots as a high-dimensional reinforcement learning (RL) problem. We unify morphology change, locomotion, and environment interaction in the same action space, and introduce an appropriate, coarse-to-fine curriculum that enables us to discover policies that accomplish fine-grained control of the resulting robots. We also introduce DittoGym, a comprehensive RL benchmark for reconfigurable soft robots that require fine-grained morphology changes to accomplish the tasks. Finally, we evaluate our proposed coarse-to-fine algorithm on DittoGym and demonstrate robots that learn to change their morphology several times within a sequence, uniquely enabled by our RL algorithm. More results are available at https://suninghuang19.github.io/dittogym_page/.