# GLSO: Grammar-guided Latent Space Optimization for Sample-efficient Robot Design Automation

arXiv: https://arxiv.org/abs/2209.11748 · HTML: https://mlsanity.com/paper/2209.11748/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 49 references that are in the map.
- First posted on arXiv 23 Sep 2022.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Grammar Variational Autoencoder](https://mlsanity.com/paper/1703.01925/) (2017)
- [Embodied Intelligence via Learning and Evolution](https://mlsanity.com/paper/2102.02202/) (2021)
- [Discriminative Embeddings of Latent Variable Models for Structured Data](https://mlsanity.com/paper/1603.05629/) (2016)
- [Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining](https://mlsanity.com/paper/2006.09191/) (2020)
- [One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control](https://mlsanity.com/paper/2007.04976/) (2020)
- [Syntax-Directed Variational Autoencoder for Structured Data](https://mlsanity.com/paper/1802.08786/) (2018)
- [Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularity](https://mlsanity.com/paper/1902.05546/) (2019)
- [Improving black-box optimization in VAE latent space using decoder uncertainty](https://mlsanity.com/paper/2107.00096/) (2021)
- [Evolution Gym: A Large-Scale Benchmark for Evolving Soft Robots](https://mlsanity.com/paper/2201.09863/) (2022)
- [Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design](https://mlsanity.com/paper/2110.03659/) (2021)
- [Neural Graph Evolution: Towards Efficient Automatic Robot Design](https://mlsanity.com/paper/1906.05370/) (2019)
- [Fit2Form: 3D Generative Model for Robot Gripper Form Design](https://mlsanity.com/paper/2011.06498/) (2020)
- [Multi-Objective Graph Heuristic Search for Terrestrial Robot Design](https://mlsanity.com/paper/2107.05858/) (2021)
- [Emergent Hand Morphology and Control from Optimizing Robust Grasps of Diverse Objects](https://mlsanity.com/paper/2012.12209/) (2020)
- [Learning Robot Structure and Motion Embeddings using Graph Neural Networks](https://mlsanity.com/paper/2109.07543/) (2021)

## Built on by (most cited first, 16 of 16)

- [OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoning](https://mlsanity.com/paper/2505.11917/) (2025)
- [Generative Models in Decision Making: A Survey](https://mlsanity.com/paper/2502.17100/) (2025)
- [Dynamics-Guided Diffusion Model for Sensor-less Robot Manipulator Design](https://mlsanity.com/paper/2402.15038/) (2024)
- [RoboMorph: Evolving Robot Morphology using Large Language Models](https://mlsanity.com/paper/2407.08626/) (2024)
- [DittoGym: Learning to Control Soft Shape-Shifting Robots](https://mlsanity.com/paper/2401.13231/) (2024)
- [Leveraging Hyperbolic Embeddings for Coarse-to-Fine Robot Design](https://mlsanity.com/paper/2311.00462/) (2023)
- [Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges](https://mlsanity.com/paper/2512.04770/) (2025)
- [CompetEvo: Towards Morphological Evolution from Competition](https://mlsanity.com/paper/2405.18300/) (2024)
- [A Design Co-Pilot for Task-Tailored Manipulators](https://mlsanity.com/paper/2509.13077/) (2025)
- [Shape Your Body: Value Gradients for Multi-Embodiment Robot Design](https://mlsanity.com/paper/2606.00702/) (2026)
- [BRIDGE: An Open-Source Humanoid Platform via Morphology-Control Co-Design for Physical AI](https://mlsanity.com/paper/2609.03497/) (2026)
- [RobotDesignGPT: Automated Robot Design Synthesis using Vision Language Models](https://mlsanity.com/paper/2601.11801/) (2026)
- [LEGO: Latent-space Exploration for Geometry-aware Optimization of Humanoid Kinematic Design](https://mlsanity.com/paper/2604.08636/) (2026)
- [IP Protection in the Era of Visual Generative AI: A Survey](https://mlsanity.com/paper/2608.14730/) (2026)
- [Shaping the Evolutionary Dynamics of Robot Morphology via Adaptive Control Learning](https://mlsanity.com/paper/2608.23100/) (2026)
- [Evolving Dexterous Robots from Scratch](https://mlsanity.com/paper/2609.33101/) (2026)

## Abstract (arXiv)

Robots have been used in all sorts of automation, and yet the design of robots remains mainly a manual task. We seek to provide design tools to automate the design of robots themselves. An important challenge in robot design automation is the large and complex design search space which grows exponentially with the number of components, making optimization difficult and sample inefficient. In this work, we present Grammar-guided Latent Space Optimization (GLSO), a framework that transforms design automation into a low-dimensional continuous optimization problem by training a graph variational autoencoder (VAE) to learn a mapping between the graph-structured design space and a continuous latent space. This transformation allows optimization to be conducted in a continuous latent space, where sample efficiency can be significantly boosted by applying algorithms such as Bayesian Optimization. GLSO guides training of the VAE using graph grammar rules and robot world space features, such that the learned latent space focus on valid robots and is easier for the optimization algorithm to explore. Importantly, the trained VAE can be reused to search for designs specialized to multiple different tasks without retraining. We evaluate GLSO by designing robots for a set of locomotion tasks in simulation, and demonstrate that our method outperforms related state-of-the-art robot design automation methods.