mlsanityData to 14 Oct 2026

Large Language Models for Robotics: A Survey

arXiv 2311.07226 ↗First posted 13 Nov 2023cs.RO cs.AIcited by 133

Not on the map. Why?

Cited by 27 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The human ability to learn, generalize, and control complex manipulation tasks through multi-modality feedback suggests a unique capability, which we refer to as dexterity intelligence. Understanding and assessing this intelligence is a complex task. Amidst the swift progress and extensive proliferation of large language models (LLMs), their applications in the field of robotics have garnered increasing attention. LLMs possess the ability to process and generate natural language, facilitating efficient interaction and collaboration with robots. Researchers and engineers in the field of robotics have recognized the immense potential of LLMs in enhancing robot intelligence, human-robot interaction, and autonomy. Therefore, this comprehensive review aims to summarize the applications of LLMs in robotics, delving into their impact and contributions to key areas such as robot control, perception, decision-making, and planning. This survey first provides an overview of the background and development of LLMs for robotics, followed by a discussion of their benefits and recent advancements in LLM-based robotic models. It then explores various techniques, employed in perception, decision-making, control, and interaction, as well as cross-module coordination in practical tasks. Finally, we review current applications of LLMs in robotics and outline potential challenges they may face in the near future. Embodied intelligence represents the future of intelligent systems, and LLM-based robotics is one of the most promising yet challenging paths toward achieving it.

Key facts. Cited by 133 papers in the map (data through 14 Oct 2026). Builds on 141 of its 291 references that are in the map. First posted on arXiv 13 Nov 2023.

Builds on

141 of 291 references.

References · 141

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2023GPT-4 Technical Reportcited by 18,396
  7. 2022Training language models to follow instructions with human feedbackcited by 15,971
  8. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  9. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  10. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  11. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  12. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
38 more
  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2022Flow Matching for Generative Modelingcited by 5,702
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  8. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  10. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  12. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  13. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  14. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  15. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  16. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  17. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  18. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  19. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  20. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  21. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  22. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  23. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  24. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  25. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  26. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  27. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  28. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  29. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  30. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  31. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  32. 2023A Survey on Multimodal Large Language Modelscited by 909
  33. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  34. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  35. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  36. 2022A Generalist Agentcited by 808
  37. 2023Large Language Modelscited by 780
  38. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766

50 most cited of 141 All 141 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 61; 2025: 49; 2026: 20.

133 papers.

Sort:

By area:

  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 634
  2. 2023Multimodal Large Language Models: A Surveycited by 195
  3. 2024Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillationcited by 166
  4. 2024Language Models with Conformal Factuality Guaranteescited by 131
  5. 2023Large Language Models in Law: A Surveycited by 128
  6. 2024On-Device Language Models: A Comprehensive Reviewcited by 104
  7. 2024If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agentscited by 79
  8. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspectivecited by 75
  9. 2024Real-World Robot Applications of Foundation Models: A Reviewcited by 61
  10. 2023Large Language Models in Education: Vision and Opportunitiescited by 54
  11. 2024Offline Training of Language Model Agents with Functions as Learnable Weightscited by 48
  12. 2024eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Datacited by 46
38 more
  1. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 43
  2. 2024BadRobot: Jailbreaking Embodied LLM Agents in the Physical Worldcited by 43
  3. 2025UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Makingcited by 42
  4. 2024Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4Vcited by 40
  5. 2024FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Modelscited by 40
  6. 2024Large Language Models for Medicine: A Surveycited by 38
  7. 2024From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Controlcited by 35
  8. 2025Safety Guardrails for LLM-Enabled Robotscited by 35
  9. 2024Large Language Models for Education: A Surveycited by 34
  10. 2025Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigationcited by 30
  11. 2024ReplanVLM: Replanning Robotic Tasks with Visual Language Modelscited by 28
  12. 2025Large Language Models for Multi-Robot Systems: A Surveycited by 27
  13. 2024Task Success is not Enough: Investigating the Use of Video-Language Models as Behavior Critics for Catching Undesirable Agent Behaviorscited by 25
  14. 2025iVISPAR -- An Interactive Visual-Spatial Reasoning Benchmark for VLMscited by 25
  15. 2024A Survey of Robotic Language Grounding: Tradeoffs between Symbols and Embeddingscited by 24
  16. 2024Large Language Models for Manufacturingcited by 24
  17. 2025Safety Aware Task Planning via Large Language Models in Roboticscited by 24
  18. 2024Watermarking Techniques for Large Language Models: A Surveycited by 23
  19. 2024DetectiveQA: Evaluating Long-Context Reasoning on Detective Novelscited by 21
  20. 2024Surveying the MLLM Landscape: A Meta-Review of Current Surveyscited by 20
  21. 2024Revealing the Barriers of Language Agents in Planningcited by 20
  22. 2025Towards Embodied Agentic AI: Review and Classification of LLM- and VLM-Driven Robot Autonomy and Interactioncited by 19
  23. 2024Probabilistically Correct Language-based Multi-Robot Planning using Conformal Predictioncited by 18
  24. 2024Large Action Models: From Inception to Implementationcited by 18
  25. 2024Hallucination Detection in Foundation Models for Decision-Making: A Flexible Definition and Review of the State of the Artcited by 17
  26. 2025TETRIS: Optimal Draft Token Selection for Batch Speculative Decodingcited by 15
  27. 2024Enhancing Investment Analysis: Optimizing AI-Agent Collaboration in Financial Researchcited by 14
  28. 2025On Almost Surely Safe Alignment of Large Language Models at Inference-Timecited by 14
  29. 2024On Pre-training of Multimodal Language Models Customized for Chart Understandingcited by 13
  30. 2025Fate: Fast Edge Inference of Mixture-of-Experts Models via Cross-Layer Gatecited by 13
  31. 2025Advancing Embodied Agent Security: From Safety Benchmarks to Input Moderationcited by 13
  32. 2024Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulationcited by 11
  33. 2025Fast and Cost-effective Speculative Edge-Cloud Decoding with Early Exitscited by 11
  34. 2024Similarity-based Neighbor Selection for Graph LLMscited by 10
  35. 2024Towards Collaborative Intelligence: Propagating Intentions and Reasoning for Multi-Agent Coordination with Large Language Modelscited by 10
  36. 2025Mixed Signals: Decoding VLMs' Reasoning and Underlying Bias in Vision-Language Conflictcited by 10
  37. 2025X-GridAgent: An LLM-Powered Agentic AI System for Assisting Power Grid Analysiscited by 10
  38. 2024A Survey on Symbolic Knowledge Distillation of Large Language Modelscited by 9

50 most cited of 133 All 133 (JSON)