mlsanityData to 7 Oct 2026

A Survey of Robotic Language Grounding: Tradeoffs between Symbols and Embeddings

arXiv 2405.13245 ↗First posted 21 May 2024cs.RO cs.AI cs.CLcited by 23

Not on the map. Why?

Abstract (arXiv)

With large language models, robots can understand language more flexibly and more capable than ever before. This survey reviews and situates recent literature into a spectrum with two poles: 1) mapping between language and some manually defined formal representation of meaning, and 2) mapping between language and high-dimensional vector spaces that translate directly to low-level robot policy. Using a formal representation allows the meaning of the language to be precisely represented, limits the size of the learning problem, and leads to a framework for interpretability and formal safety guarantees. Methods that embed language and perceptual data into high-dimensional spaces avoid this manually specified symbolic structure and thus have the potential to be more general when fed enough data but require more data and computing to train. We discuss the benefits and tradeoffs of each approach and finish by providing directions for future work that achieves the best of both worlds.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 53 of its 84 references that are in the map. First posted on arXiv 21 May 2024.

Builds on

53 of 84 references.

References · 53

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  6. 2022Emergent Abilities of Large Language Modelscited by 2,483
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  9. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  10. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  11. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  12. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
38 more
  1. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  2. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  3. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  4. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  5. 2022A Generalist Agentcited by 802
  6. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  7. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  8. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  9. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  10. 2020Scaling Laws for Autoregressive Generative Modelingcited by 510
  11. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  12. 2023Learning Interactive Real-World Simulatorscited by 440
  13. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  14. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 372
  15. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
  16. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  17. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 308
  18. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 264
  19. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 220
  20. 2023Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planningcited by 204
  21. 2023Video Language Planningcited by 160
  22. 2022Safe Control with Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction methodscited by 148
  23. 2023Generalized Planning in PDDL Domains with Pretrained Large Language Modelscited by 139
  24. 2023Translating Natural Language to Planning Goals with Large-Language Modelscited by 132
  25. 2023Large Language Models for Robotics: A Surveycited by 132
  26. 2023AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkerscited by 111
  27. 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectivescited by 88
  28. 2023Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agentscited by 67
  29. 2023NL2TL: Transforming Natural Languages to Temporal Logics using Large Language Modelscited by 57
  30. 2022Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning taskscited by 56
  31. 2023Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environmentscited by 52
  32. 2015From Pixels to Torques: Policy Learning with Deep Dynamical Modelscited by 44
  33. 2022Temporally Consistent Transformers for Video Generationcited by 43
  34. 2023Data-Efficient Learning of Natural Language to Linear Temporal Logic Translators for Robot Task Specificationcited by 38
  35. 2023Interactive Task Planning with Language Modelscited by 36
  36. 2023What's Left? Concept Grounding with Logic-Enhanced Foundation Modelscited by 33
  37. 2021Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recoverycited by 32
  38. 2020Learning a natural-language to LTL executable semantic parser for grounded roboticscited by 22

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 12; 2026: 6.

23 papers.

Sort:

By area:

  1. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 20
  2. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  3. 2024Pragmatic Embodied Spoken Instruction Following in Human-Robot Collaboration with Theory of Mindcited by 6
  4. 2024The Duality of Generative AI and Reinforcement Learning in Robotics: A Reviewcited by 5
  5. 2025Free-T2M: Robust Text-to-Motion Generation for Humanoid Robots via Frequency-Domaincited by 5
  6. 2025Physically Ground Commonsense Knowledge for Articulated Object Manipulation with Analytic Conceptscited by 5
  7. 2025Understanding Common Ground Misalignment in Goal-Oriented Dialog: A Case-Study with Ubuntu Chat Logscited by 4
  8. 2025Prior Lessons of Incremental Dialogue and Robot Action Management for the Age of Language Modelscited by 2
  9. 2025HyCodePolicy: Hybrid Language Controllers for Multimodal Monitoring and Decision in Embodied Agentscited by 2
  10. 2026Offload or Overload: A Platform Measurement Study of Mobile Robotic Manipulation Workloadscited by 2
  11. 2024Instructional Text Across Disciplines: A Survey of Representations, Downstream Tasks, and Open Challenges Toward Capable AI Agentscited by 1
  12. 2025GBPP: Grasp-Aware Base Placement Prediction for Robots via Two-Stage Learningcited by 1
11 more
  1. 2026Meanings and Measurements: Multi-Agent Probabilistic Grounding for Vision-Language Navigationcited by 1
  2. 2026Structural-Ambiguity-Aware Translation from Natural Language to Signal Temporal Logiccited by 1
  3. 2026Comprehensive AI governance requires addressing non-model gainscited by 1
  4. 2026SpatialUAV: Benchmarking Spatial Intelligence for Low-Altitude UAV Perception, Collaboration, and Motioncited by 1
  5. 2024Nl2Hltl2Plan: Scaling Up Natural Language Understanding for Multi-Robots Through Hierarchical Temporal Logic Task Representationcited by 0
  6. 2025FRIDA to the Rescue! Analyzing Synthetic Data Effectiveness in Object-Based Common Sense Reasoning for Disaster Responsecited by 0
  7. 2025A Signal Contract for Online Language Grounding and Discovery in Decision-Makingcited by 0
  8. 2025Challenges in Grounding Language in the Real Worldcited by 0
  9. 2025Using Natural Language for Human-Robot Collaboration in the Real Worldcited by 0
  10. 2025Prompting Robot Teams with Natural Languagecited by 0
  11. 2026Bounding Boxes as Goals: Language-Conditioned Grasping via Neuro-Symbolic Planningcited by 0