# SayTap: Language to Quadrupedal Locomotion

arXiv: https://arxiv.org/abs/2306.07580 · HTML: https://mlsanity.com/paper/2306.07580/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2023.

## Built on by (most cited first, 30 of 41)

- [Vision Language Models are In-Context Value Learners](https://mlsanity.com/paper/2411.04549/) (2024)
- [DrEureka: Language Model Guided Sim-To-Real Transfer](https://mlsanity.com/paper/2406.01967/) (2024)
- [HiRT: Enhancing Robotic Control with Hierarchical Robot Transformers](https://mlsanity.com/paper/2410.05273/) (2024)
- [Real-World Robot Applications of Foundation Models: A Review](https://mlsanity.com/paper/2402.05741/) (2024)
- [QUAR-VLA: Vision-Language-Action Model for Quadruped Robots](https://mlsanity.com/paper/2312.14457/) (2023)
- [Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks](https://mlsanity.com/paper/2405.01534/) (2024)
- [Prompt a Robot to Walk with Large Language Models](https://mlsanity.com/paper/2309.09969/) (2023)
- [What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Models](https://mlsanity.com/paper/2512.03422/) (2025)
- [Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learning](https://mlsanity.com/paper/2310.20587/) (2023)
- [Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web](https://mlsanity.com/paper/2311.18751/) (2023)
- [Eurekaverse: Environment Curriculum Generation via Large Language Models](https://mlsanity.com/paper/2411.01775/) (2024)
- [Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models](https://mlsanity.com/paper/2407.02666/) (2024)
- [MobileVLA-R1: Reinforcing Vision-Language-Action for Mobile Robots](https://mlsanity.com/paper/2511.17889/) (2025)
- [The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learning](https://mlsanity.com/paper/2502.15214/) (2025)
- [Video2Reward: Generating Reward Function from Videos for Legged Robot Behavior Learning](https://mlsanity.com/paper/2412.05515/) (2024)
- [QUART-Online: Latency-Free Large Multimodal Language Model for Quadruped Robot Learning](https://mlsanity.com/paper/2412.15576/) (2024)
- [Helpful DoggyBot: Open-World Object Fetching using Legged Robots and Vision-Language Models](https://mlsanity.com/paper/2410.00231/) (2024)
- [HYPERmotion: Learning Hybrid Behavior Planning for Autonomous Loco-manipulation](https://mlsanity.com/paper/2406.14655/) (2024)
- [ROVER: Recursive Reasoning Over Videos with Vision-Language Models for Embodied Tasks](https://mlsanity.com/paper/2508.01943/) (2025)
- [Vision Language Models Can Parse Floor Plan Maps](https://mlsanity.com/paper/2409.12842/) (2024)
- [SwarmGPT: Combining Large Language Models with Safe Motion Planning for Drone Swarm Choreography](https://mlsanity.com/paper/2412.08428/) (2024)
- [Natural Language as Policies: Reasoning for Coordinate-Level Embodied Control with LLMs](https://mlsanity.com/paper/2403.13801/) (2024)
- [TalkWithMachines: Enhancing Human-Robot Interaction for Interpretable Industrial Robotics Through Large/Vision Language Models](https://mlsanity.com/paper/2412.15462/) (2024)
- [Taking Flight with Dialogue: Enabling Natural Language Control for PX4-based Drone Agent](https://mlsanity.com/paper/2506.07509/) (2025)
- [BBSEA: An Exploration of Brain-Body Synchronization for Embodied Agents](https://mlsanity.com/paper/2402.08212/) (2024)
- [LLM-Mediated Guidance of MARL Systems](https://mlsanity.com/paper/2503.13553/) (2025)
- [Meta-Control: Automatic Model-based Control Synthesis for Heterogeneous Robot Skills](https://mlsanity.com/paper/2405.11380/) (2024)
- [LAMS: LLM-Driven Automatic Mode Switching for Assistive Teleoperation](https://mlsanity.com/paper/2501.08558/) (2025)
- [PIP-LLM: Integrating PDDL-Integer Programming with LLMs for Coordinating Multi-Robot Teams Using Natural Language](https://mlsanity.com/paper/2510.22784/) (2025)
- [A Prompt-driven Task Planning Method for Multi-drones based on Large Language Model](https://mlsanity.com/paper/2406.00006/) (2024)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated the potential to perform high-level planning. Yet, it remains a challenge for LLMs to comprehend low-level commands, such as joint angle targets or motor torques. This paper proposes an approach to use foot contact patterns as an interface that bridges human commands in natural language and a locomotion controller that outputs these low-level commands. This results in an interactive system for quadrupedal robots that allows the users to craft diverse locomotion behaviors flexibly. We contribute an LLM prompt design, a reward function, and a method to expose the controller to the feasible distribution of contact patterns. The results are a controller capable of achieving diverse locomotion patterns that can be transferred to real robot hardware. Compared with other design choices, the proposed approach enjoys more than 50% success rate in predicting the correct contact patterns and can solve 10 more tasks out of a total of 30 tasks. Our project site is: https://saytap.github.io.