# On the Exploration of LM-Based Soft Modular Robot Design

arXiv: https://arxiv.org/abs/2411.00345 · HTML: https://mlsanity.com/paper/2411.00345/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 32 references that are in the map.
- First posted on arXiv 1 Nov 2024.

## Builds on (most cited first, 7 of 7)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [DiffTaichi: Differentiable Programming for Physical Simulation](https://mlsanity.com/paper/1910.00935/) (2019)
- [LangSuitE: Planning, Controlling and Interacting with Large Language Models in Embodied Text Environments](https://mlsanity.com/paper/2406.16294/) (2024)
- [Bio-inspired Tensegrity Soft Modular Robots](https://mlsanity.com/paper/1703.10139/) (2017)
- [Soft Lattice Modules that Behave Independently and Collectively](https://mlsanity.com/paper/2110.11485/) (2021)
- [SoftSnap: Rapid Prototyping of Untethered Soft Robots Using Snap-Together Modules](https://mlsanity.com/paper/2410.19169/) (2024)
- [Into the Unknown: Generating Geospatial Descriptions for New Environments](https://mlsanity.com/paper/2406.19967/) (2024)

## Built on by (most cited first, 1 of 1)

- [RobotDesignGPT: Automated Robot Design Synthesis using Vision Language Models](https://mlsanity.com/paper/2601.11801/) (2026)

## Abstract (arXiv)

Recent large language models (LLMs) have demonstrated promising capabilities in modeling real-world knowledge and enhancing knowledge-based generation tasks. In this paper, we further explore the potential of using LLMs to aid in the design of soft modular robots, taking into account both user instructions and physical laws, to reduce the reliance on extensive trial-and-error experiments typically needed to achieve robot designs that meet specific structural or task requirements. Specifically, we formulate the robot design process as a sequence generation task and find that LLMs are able to capture key requirements expressed in natural language and reflect them in the construction sequences of robots. To simplify, rather than conducting real-world experiments to assess design quality, we utilize a simulation tool to provide feedback to the generative model, allowing for iterative improvements without requiring extensive human annotations. Furthermore, we introduce five evaluation metrics to assess the quality of robot designs from multiple angles including task completion and adherence to instructions, supporting an automatic evaluation process. Our model performs well in evaluations for designing soft modular robots with uni- and bi-directional locomotion and stair-descending capabilities, highlighting the potential of using natural language and LLMs for robot design. However, we also observe certain limitations that suggest areas for further improvement.