# Bridging Language and Action: A Survey of Language-Conditioned Robot Manipulation

arXiv: https://arxiv.org/abs/2312.10807 · HTML: https://mlsanity.com/paper/2312.10807/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2023.

## Built on by (most cited first, 22 of 22)

- [Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey](https://mlsanity.com/paper/2508.13073/) (2025)
- [Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning](https://mlsanity.com/paper/2508.16949/) (2025)
- [When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models](https://mlsanity.com/paper/2405.10255/) (2024)
- [BlenderAlchemy: Editing 3D Graphics with Vision-Language Models](https://mlsanity.com/paper/2404.17672/) (2024)
- [Details Make a Difference: Object State-Sensitive Neurorobotic Task Planning](https://mlsanity.com/paper/2406.09988/) (2024)
- [Seeing and Knowing in the Wild: Open-domain Visual Entity Recognition with Large-scale Knowledge Graphs via Contrastive Learning](https://mlsanity.com/paper/2510.13675/) (2025)
- [Predicting the Road Ahead: A Knowledge Graph based Foundation Model for Scene Understanding in Autonomous Driving](https://mlsanity.com/paper/2503.18730/) (2025)
- [Robotic Long-Horizon Manipulation with Progressive In-Context Code Generation and Episodic Feedback](https://mlsanity.com/paper/2503.21969/) (2025)
- [Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Review](https://mlsanity.com/paper/2505.20503/) (2025)
- [Bridging Perception and Action: Spatially-Grounded Mid-Level Representations for Robot Generalization](https://mlsanity.com/paper/2506.06196/) (2025)
- [Bi-VLA: Bilateral Control-Based Imitation Learning via Vision-Language Fusion for Action Generation](https://mlsanity.com/paper/2509.18865/) (2025)
- [Dreaming the Unseen: World Model-regularized Diffusion Policy for Out-of-Distribution Robustness](https://mlsanity.com/paper/2603.21017/) (2026)
- [RoboInspector: Unveiling the Unreliability of Policy Code for LLM-enabled Robotic Manipulation](https://mlsanity.com/paper/2508.21378/) (2025)
- [Inference-stage Adaptation-projection Strategy Adapts Diffusion Policy to Cross-manipulators Scenarios](https://mlsanity.com/paper/2509.11621/) (2025)
- [LEMMo-Plan: LLM-Enhanced Learning from Multi-Modal Demonstration for Planning Sequential Contact-Rich Manipulation Tasks](https://mlsanity.com/paper/2409.11863/) (2024)
- [From Goal-Conditioned to Language-Conditioned Agents via Vision-Language Models](https://mlsanity.com/paper/2409.16024/) (2024)
- [DiffDef: A Diffusion Model for Generating Multimodal Goal Shapes From Demonstrations for Deformable Object Manipulation](https://mlsanity.com/paper/2506.18779/) (2025)
- [Defect-aware Hybrid Prompt Optimization via Progressive Tuning for Zero-Shot Multi-type Anomaly Detection and Segmentation](https://mlsanity.com/paper/2512.09446/) (2025)
- [Speech2Grasp: Data-Efficient Transfer of Text-Conditioned Grasp Detection to Speech in Humanoid Robots](https://mlsanity.com/paper/2607.26567/) (2026)
- [Robots That Take Initiative: A Framework for Building and Evaluating Proactive Robots](https://mlsanity.com/paper/2609.28910/) (2026)
- [Decoupled Early Exits for Task-Dependent Compute Allocation in Flow-Matching VLAs](https://mlsanity.com/paper/2609.29382/) (2026)
- [Training Generalist Navigation Policies via Action Self-Distillation](https://mlsanity.com/paper/2610.17593/) (2026)

## Abstract (arXiv)

Language-conditioned robot manipulation is an emerging field aimed at enabling seamless communication and cooperation between humans and robotic agents by teaching robots to comprehend and execute instructions conveyed in natural language. This interdisciplinary area integrates scene understanding, language processing, and policy learning to bridge the gap between human instructions and robot actions. In this comprehensive survey, we systematically explore recent advancements in language-conditioned robot manipulation. We categorize existing methods based on the primary ways language is integrated into the robot system, namely language for state evaluation, language as a policy condition, language for cognitive planning and reasoning, and language in unified vision-language-action models. Specifically, we further analyze state-of-the-art techniques from five axes of action granularity, data and supervision regimes, system cost and latency, environments and evaluations, and task specification. Additionally, we highlight the key debates in the field. Finally, we discuss open challenges and future research directions, focusing on potentially enhancing generalization capabilities and addressing safety issues in language-conditioned robot manipulators.