# Evolution and learning in differentiable robots

arXiv: https://arxiv.org/abs/2405.14712 · HTML: https://mlsanity.com/paper/2405.14712/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 65 references that are in the map.
- First posted on arXiv 23 May 2024.

## Builds on (most cited first, 21 of 21)

- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [DiffTaichi: Differentiable Programming for Physical Simulation](https://mlsanity.com/paper/1910.00935/) (2019)
- [A Differentiable Physics Engine for Deep Learning in Robotics](https://mlsanity.com/paper/1611.01652/) (2016)
- [Accelerated Policy Learning with Parallel Differentiable Simulation](https://mlsanity.com/paper/2204.07137/) (2022)
- [ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics](https://mlsanity.com/paper/1810.01054/) (2018)
- [Weight Agnostic Neural Networks](https://mlsanity.com/paper/1906.04358/) (2019)
- [Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularity](https://mlsanity.com/paper/1902.05546/) (2019)
- [Evolution Gym: A Large-Scale Benchmark for Evolving Soft Robots](https://mlsanity.com/paper/2201.09863/) (2022)
- [Scalable Co-Optimization of Morphology and Control in Embodied Machines](https://mlsanity.com/paper/1706.06133/) (2017)
- [Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design](https://mlsanity.com/paper/2110.03659/) (2021)
- [Neural Graph Evolution: Towards Efficient Automatic Robot Design](https://mlsanity.com/paper/1906.05370/) (2019)
- [Task-Agnostic Morphology Evolution](https://mlsanity.com/paper/2102.13100/) (2021)
- [Efficient automatic design of robots](https://mlsanity.com/paper/2306.03263/) (2023)
- [How morphological development can guide evolution](https://mlsanity.com/paper/1711.07387/) (2017)
- [Scalable sim-to-real transfer of soft robot designs](https://mlsanity.com/paper/1911.10290/) (2019)
- [Soft Robots Learn to Crawl: Jointly Optimizing Design and Control with Sim-to-Real Transfer](https://mlsanity.com/paper/2202.04575/) (2022)
- [Reinforcement learning for freeform robot design](https://mlsanity.com/paper/2310.05670/) (2023)
- [Evolving soft locomotion in aquatic and terrestrial environments: effects of material properties and environmental transitions](https://mlsanity.com/paper/1711.06605/) (2017)
- [4D topology optimization: Integrated optimization of the structure and self-actuation of soft bodies for dynamic motions](https://mlsanity.com/paper/2302.00905/) (2023)
- [Scale invariant robot behavior with fractals](https://mlsanity.com/paper/2103.04876/) (2021)
- [Interoceptive robustness through environment-mediated morphological development](https://mlsanity.com/paper/1804.02257/) (2018)

## Built on by (most cited first, 5 of 5)

- [Text2Robot: Evolutionary Robot Design from Text Descriptions](https://mlsanity.com/paper/2406.19963/) (2024)
- [Accelerated co-design of robots through morphological pretraining](https://mlsanity.com/paper/2502.10862/) (2025)
- [Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges](https://mlsanity.com/paper/2512.04770/) (2025)
- [Generative Evolutionary Design of Voxel-Based Soft Robots with Provable Optimality](https://mlsanity.com/paper/2609.29491/) (2026)
- [Evolving Dexterous Robots from Scratch](https://mlsanity.com/paper/2609.33101/) (2026)

## Abstract (arXiv)

The automatic design of robots has existed for 30 years but has been constricted by serial non-differentiable design evaluations, premature convergence to simple bodies or clumsy behaviors, and a lack of sim2real transfer to physical machines. Thus, here we employ massively-parallel differentiable simulations to rapidly and simultaneously optimize individual neural control of behavior across a large population of candidate body plans and return a fitness score for each design based on the performance of its fully optimized behavior. Non-differentiable changes to the mechanical structure of each robot in the population -- mutations that rearrange, combine, add, or remove body parts -- were applied by a genetic algorithm in an outer loop of search, generating a continuous flow of novel morphologies with highly-coordinated and graceful behaviors honed by gradient descent. This enabled the exploration of several orders-of-magnitude more designs than all previous methods, despite the fact that robots here have the potential to be much more complex, in terms of number of independent motors, than those in prior studies. We found that evolution reliably produces ``increasingly differentiable'' robots: body plans that smooth the loss landscape in which learning operates and thereby provide better training paths toward performant behaviors. Finally, one of the highly differentiable morphologies discovered in simulation was realized as a physical robot and shown to retain its optimized behavior. This provides a cyberphysical platform to investigate the relationship between evolution and learning in biological systems and broadens our understanding of how a robot's physical structure can influence the ability to train policies for it. Videos and code at https://sites.google.com/view/eldir.