# Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design

arXiv: https://arxiv.org/abs/2110.03659 · HTML: https://mlsanity.com/paper/2110.03659/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 7 Oct 2021.

## Builds on (most cited first, 23 of 23)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks](https://mlsanity.com/paper/1810.02244/) (2018)
- [One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control](https://mlsanity.com/paper/2007.04976/) (2020)
- [Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularity](https://mlsanity.com/paper/1902.05546/) (2019)
- [My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Control](https://mlsanity.com/paper/2010.01856/) (2020)
- [Reinforcement Learning for Improving Agent Design](https://mlsanity.com/paper/1810.03779/) (2018)
- [Jointly Learning to Construct and Control Agents using Deep Reinforcement Learning](https://mlsanity.com/paper/1801.01432/) (2018)
- [Policy Transfer with Strategy Optimization](https://mlsanity.com/paper/1810.05751/) (2018)
- [Scalable Co-Optimization of Morphology and Control in Embodied Machines](https://mlsanity.com/paper/1706.06133/) (2017)
- [Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning](https://mlsanity.com/paper/1911.06832/) (2019)
- [Neural Graph Evolution: Towards Efficient Automatic Robot Design](https://mlsanity.com/paper/1906.05370/) (2019)
- [Hardware as Policy: Mechanical and Computational Co-Optimization using Deep Reinforcement Learning](https://mlsanity.com/paper/2008.04460/) (2020)
- [Contrasting Exploration in Parameter and Action Space: A Zeroth-Order Optimization Perspective](https://mlsanity.com/paper/1901.11503/) (2019)
- [SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learning](https://mlsanity.com/paper/2101.06005/) (2021)
- [Task-Agnostic Morphology Evolution](https://mlsanity.com/paper/2102.13100/) (2021)
- [Policy Transfer via Kinematic Domain Randomization and Adaptation](https://mlsanity.com/paper/2011.01891/) (2020)
- [Interactive Co-Design of Form and Function for Legged Robots using the Adjoint Method](https://mlsanity.com/paper/1801.00385/) (2018)

## Built on by (most cited first, 30 of 38)

- [Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability](https://mlsanity.com/paper/2209.08025/) (2022)
- [Universal Morphology Control via Contextual Modulation](https://mlsanity.com/paper/2302.11070/) (2023)
- [GLSO: Grammar-guided Latent Space Optimization for Sample-efficient Robot Design Automation](https://mlsanity.com/paper/2209.11748/) (2022)
- [Efficient automatic design of robots](https://mlsanity.com/paper/2306.03263/) (2023)
- [A System for Morphology-Task Generalization via Unified Representation and Behavior Distillation](https://mlsanity.com/paper/2211.14296/) (2022)
- [Symmetry-Aware Robot Design with Structured Subgroups](https://mlsanity.com/paper/2306.00036/) (2023)
- [From Universal Humanoid Control to Automatic Physically Valid Character Creation](https://mlsanity.com/paper/2206.09286/) (2022)
- [VLMgineer: Vision Language Models as Robotic Toolsmiths](https://mlsanity.com/paper/2507.12644/) (2025)
- [BodyGen: Advancing Towards Efficient Embodiment Co-Design](https://mlsanity.com/paper/2503.00533/) (2025)
- [HERD: Continuous Human-to-Robot Evolution for Learning from Human Demonstration](https://mlsanity.com/paper/2212.04359/) (2022)
- [MORPH: Design Co-optimization with Reinforcement Learning via a Differentiable Hardware Model Proxy](https://mlsanity.com/paper/2309.17227/) (2023)
- [Reinforcement learning for freeform robot design](https://mlsanity.com/paper/2310.05670/) (2023)
- [Learning to Design and Use Tools for Robotic Manipulation](https://mlsanity.com/paper/2311.00754/) (2023)
- [DittoGym: Learning to Control Soft Shape-Shifting Robots](https://mlsanity.com/paper/2401.13231/) (2024)
- [Accelerated co-design of robots through morphological pretraining](https://mlsanity.com/paper/2502.10862/) (2025)
- [Curriculum Reinforcement Learning via Morphology-Environment Co-Evolution](https://mlsanity.com/paper/2309.12529/) (2023)
- [Leveraging Hyperbolic Embeddings for Coarse-to-Fine Robot Design](https://mlsanity.com/paper/2311.00462/) (2023)
- [Evolution and learning in differentiable robots](https://mlsanity.com/paper/2405.14712/) (2024)
- [House of Dextra: Cross-embodied Co-design for Dexterous Hands](https://mlsanity.com/paper/2512.03743/) (2025)
- [Task2Morph: Differentiable Task-inspired Framework for Contact-Aware Robot Design](https://mlsanity.com/paper/2403.19093/) (2024)
- [Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges](https://mlsanity.com/paper/2512.04770/) (2025)
- [Efficient Morphology-Control Co-Design via Stackelberg Proximal Policy Optimization](https://mlsanity.com/paper/2603.15388/) (2026)
- [The Introspective Agent: Interdependence of Strategy, Physiology, and Sensing for Embodied Agents](https://mlsanity.com/paper/2201.00411/) (2022)
- [ECoDe: A Sample-Efficient Method for Co-Design of Robotic Agents](https://mlsanity.com/paper/2309.04085/) (2023)
- [CompetEvo: Towards Morphological Evolution from Competition](https://mlsanity.com/paper/2405.18300/) (2024)
- [Toward Humanoid Brain-Body Co-design: Joint Optimization of Control and Morphology for Fall Recovery](https://mlsanity.com/paper/2510.22336/) (2025)
- [Shape Your Body: Value Gradients for Multi-Embodiment Robot Design](https://mlsanity.com/paper/2606.00702/) (2026)
- [Designing Tools with Control Confidence](https://mlsanity.com/paper/2510.12630/) (2025)
- [Latent Diffeomorphic Co-Design of End-Effectors for Deformable and Fragile Object Manipulation](https://mlsanity.com/paper/2602.17921/) (2026)
- [Unified Simulation, Perception, and Generation of Human Behavior](https://mlsanity.com/paper/2204.13678/) (2022)

## Abstract (arXiv)

An agent's functionality is largely determined by its design, i.e., skeletal structure and joint attributes (e.g., length, size, strength). However, finding the optimal agent design for a given function is extremely challenging since the problem is inherently combinatorial and the design space is prohibitively large. Additionally, it can be costly to evaluate each candidate design which requires solving for its optimal controller. To tackle these problems, our key idea is to incorporate the design procedure of an agent into its decision-making process. Specifically, we learn a conditional policy that, in an episode, first applies a sequence of transform actions to modify an agent's skeletal structure and joint attributes, and then applies control actions under the new design. To handle a variable number of joints across designs, we use a graph-based policy where each graph node represents a joint and uses message passing with its neighbors to output joint-specific actions. Using policy gradient methods, our approach enables joint optimization of agent design and control as well as experience sharing across different designs, which improves sample efficiency substantially. Experiments show that our approach, Transform2Act, outperforms prior methods significantly in terms of convergence speed and final performance. Notably, Transform2Act can automatically discover plausible designs similar to giraffes, squids, and spiders. Code and videos are available at https://sites.google.com/view/transform2act.