mlsanityData to 14 Oct 2026

Text2Robot: Evolutionary Robot Design from Text Descriptions

arXiv 2406.19963 ↗First posted 28 Jun 2024cs.RO cs.AI cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Robot design has traditionally been costly and labor-intensive. Despite advancements in automated processes, it remains challenging to navigate a vast design space while producing physically manufacturable robots. We introduce Text2Robot, a framework that converts user text specifications and performance preferences into physical quadrupedal robots. Within minutes, Text2Robot can use text-to-3D models to provide strong initializations of diverse morphologies. Within a day, our geometric processing algorithms and body-control co-optimization produce a walking robot by explicitly considering real-world electronics and manufacturability. Text2Robot enables rapid prototyping and opens new opportunities for robot design with generative models.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 34 of its 66 references that are in the map. First posted on arXiv 28 Jun 2024.

Builds on

34 of 66 references.

References · 34

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  4. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  5. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  6. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  7. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  8. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  9. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  10. 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 382
  11. 2021DeepCAD: A Deep Generative Network for Computer-Aided Design Modelscited by 271
  12. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 259
22 more
  1. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  2. 2018Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddingscited by 169
  3. 2021Embodied Intelligence via Learning and Evolutioncited by 163
  4. 2023Generative AIcited by 120
  5. 2021Deep Generative Models in Engineering Design: A Reviewcited by 105
  6. 2018The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communitiescited by 78
  7. 2018Jointly Learning to Construct and Control Agents using Deep Reinforcement Learningcited by 55
  8. 2021An End-to-End Differentiable Framework for Contact-Aware Robot Designcited by 54
  9. 2019DeepGait: Planning and Control of Quadrupedal Gaits using Deep Reinforcement Learningcited by 50
  10. 20223DALL-E: Integrating Text-to-Image AI in 3D Design Workflowscited by 41
  11. 2023How Can Large Language Models Help Humans in Design and Manufacturing?cited by 41
  12. 2019Neural Graph Evolution: Towards Efficient Automatic Robot Designcited by 36
  13. 2020Hardware as Policy: Mechanical and Computational Co-Optimization using Deep Reinforcement Learningcited by 32
  14. 2024A Comprehensive Survey on 3D Content Generationcited by 28
  15. 2018A Generative Model for Inverse Design of Metamaterialscited by 27
  16. 2023Efficient automatic design of robotscited by 14
  17. 2020Representation-Free Model Predictive Control for Dynamic Motions in Quadrupedscited by 13
  18. 2023HyperFields: Towards Zero-Shot Generation of NeRFs from Textcited by 13
  19. 2016An Efficient Optimal Planning and Control Framework For Quadrupedal Locomotioncited by 12
  20. 2022Soft Robots Learn to Crawl: Jointly Optimizing Design and Control with Sim-to-Real Transfercited by 10
  21. 2023MORPH: Design Co-optimization with Reinforcement Learning via a Differentiable Hardware Model Proxycited by 8
  22. 2024Evolution and learning in differentiable robotscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 3.

8 papers.

  1. 2025VLMgineer: Vision Language Models as Robotic Toolsmithscited by 10
  2. 2024RoboMorph: Evolving Robot Morphology using Large Language Modelscited by 8
  3. 2025Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challengescited by 4
  4. 2026Latent Diffeomorphic Co-Design of End-Effectors for Deformable and Fragile Object Manipulationcited by 1
  5. 2025An integrated process for design and control of lunar robotics using AI and simulationcited by 0
  6. 2025Debate2Create: Robot Co-design via Multi-Agent LLM Debatecited by 0
  7. 2026RobotDesignGPT: Automated Robot Design Synthesis using Vision Language Modelscited by 0
  8. 2026When Search Becomes Memory: Accelerating Robot Design Discovery with Self-Evolving Skillscited by 0