# From Text to Motion: Grounding GPT-4 in a Humanoid Robot "Alter3"

arXiv: https://arxiv.org/abs/2312.06571 · HTML: https://mlsanity.com/paper/2312.06571/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2023.

## Built on by (most cited first, 13 of 13)

- [Harmon: Whole-Body Motion Generation of Humanoid Robots from Language Descriptions](https://mlsanity.com/paper/2410.12773/) (2024)
- [Language-Driven Interactive Traffic Trajectory Generation](https://mlsanity.com/paper/2405.15388/) (2024)
- [Generative Emergent Communication: Large Language Model is a Collective World Model](https://mlsanity.com/paper/2501.00226/) (2024)
- [Spontaneous Emergence of Agent Individuality through Social Interactions in LLM-Based Communities](https://mlsanity.com/paper/2411.03252/) (2024)
- [AI with Emotions: Exploring Emotional Expressions in Large Language Models](https://mlsanity.com/paper/2504.14706/) (2025)
- [From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Models](https://mlsanity.com/paper/2407.09502/) (2024)
- [Comparing Apples to Oranges: LLM-powered Multimodal Intention Prediction in an Object Categorization Task](https://mlsanity.com/paper/2404.08424/) (2024)
- [Scalable, Training-Free Visual Language Robotics: A Modular Multi-Model Framework for Consumer-Grade GPUs](https://mlsanity.com/paper/2502.01071/) (2025)
- [The Future of Intelligent Healthcare: A Systematic Analysis and Discussion on the Integration and Impact of Robots Using Large Language Models for Healthcare](https://mlsanity.com/paper/2411.03287/) (2024)
- [A Concurrent Modular Agent: Framework for Autonomous LLM Agents](https://mlsanity.com/paper/2508.19042/) (2025)
- [Proprioception Enhances Vision Language Model in Generating Captions and Subtask Segmentations for Robot Task](https://mlsanity.com/paper/2512.20876/) (2025)
- [Plantbot: Integrating Plant and Robot through LLM Modular Agent Networks](https://mlsanity.com/paper/2509.05338/) (2025)
- [Event-T2M: Event-level Conditioning for Complex Text-to-Motion Synthesis](https://mlsanity.com/paper/2602.04292/) (2026)

## Abstract (arXiv)

We report the development of Alter3, a humanoid robot capable of generating spontaneous motion using a Large Language Model (LLM), specifically GPT-4. This achievement was realized by integrating GPT-4 into our proprietary android, Alter3, thereby effectively grounding the LLM with Alter's bodily movement. Typically, low-level robot control is hardware-dependent and falls outside the scope of LLM corpora, presenting challenges for direct LLM-based robot control. However, in the case of humanoid robots like Alter3, direct control is feasible by mapping the linguistic expressions of human actions onto the robot's body through program code. Remarkably, this approach enables Alter3 to adopt various poses, such as a 'selfie' stance or 'pretending to be a ghost,' and generate sequences of actions over time without explicit programming for each body part. This demonstrates the robot's zero-shot learning capabilities. Additionally, verbal feedback can adjust poses, obviating the need for fine-tuning. A video of Alter3's generated motions is available at https://tnoinkwms.github.io/ALTER-LLM/