mlsanityData to 14 Oct 2026

Safety Control of Service Robots with LLMs and Embodied Knowledge Graphs

arXiv 2405.17846 ↗First posted 28 May 2024cs.RO cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Safety limitations in service robotics across various industries have raised significant concerns about the need for robust mechanisms ensuring that robots adhere to safe practices, thereby preventing actions that might harm humans or cause property damage. Despite advances, including the integration of Knowledge Graphs (KGs) with Large Language Models (LLMs), challenges in ensuring consistent safety in autonomous robot actions persist. In this paper, we propose a novel integration of Large Language Models with Embodied Robotic Control Prompts (ERCPs) and Embodied Knowledge Graphs (EKGs) to enhance the safety framework for service robots. ERCPs are designed as predefined instructions that ensure LLMs generate safe and precise responses. These responses are subsequently validated by EKGs, which provide a comprehensive knowledge base ensuring that the actions of the robot are continuously aligned with safety protocols, thereby promoting safer operational practices in varied contexts. Our experimental setup involved diverse real-world tasks, where robots equipped with our framework demonstrated significantly higher compliance with safety standards compared to traditional methods. This integration fosters secure human-robot interactions and positions our methodology at the forefront of AI-driven safety innovations in service robotics.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 38 of its 77 references that are in the map. First posted on arXiv 28 May 2024.

Builds on

38 of 77 references.

References · 38

  1. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  2. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  3. 2022Visual Prompt Tuningcited by 1,681
  4. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
  5. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  6. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  7. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 982
  8. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 876
  9. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  10. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
  11. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 619
  12. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
26 more
  1. 2021QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answeringcited by 388
  2. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 299
  3. 20196-DOF GraspNet: Variational Grasp Generation for Object Manipulationcited by 242
  4. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 221
  5. 2019KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoningcited by 216
  6. 2023REFLECT: Summarizing Robot Experiences for Failure Explanation and Correctioncited by 175
  7. 2022GreaseLM: Graph REASoning Enhanced Language Models for Question Answeringcited by 171
  8. 2021OpenPrompt: An Open-source Framework for Prompt-learningcited by 148
  9. 2023Task and Motion Planning with Large Language Models for Object Rearrangementcited by 141
  10. 2023LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agentcited by 135
  11. 2020Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answeringcited by 122
  12. 2019PyTorch-BigGraph: A Large-scale Graph Embedding Systemcited by 119
  13. 2022A Survey of Knowledge Enhanced Pre-trained Language Modelscited by 76
  14. 2023Knowledge Graphs: Opportunities and Challengescited by 70
  15. 2023Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agentscited by 69
  16. 2018Improving Natural Language Inference Using External Knowledge in the Science Questions Domaincited by 48
  17. 2021JointLK: Joint Reasoning with Language Models and Knowledge Graphs for Commonsense Question Answeringcited by 40
  18. 2023Deploying and Evaluating LLMs to Program Service Mobile Robotscited by 36
  19. 2020Object Handovers: a Review for Roboticscited by 28
  20. 2021LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Promptingcited by 27
  21. 2024LLM-based Robot Task Planning with Exceptional Handling for General Purpose Service Robotscited by 16
  22. 2014RoboBrain: Large-Scale Knowledge Engine for Robotscited by 13
  23. 2021Patterns for Representing Knowledge Graphs to Communicate Situational Knowledge of Service Robotscited by 7
  24. 2023Self-Recovery Prompting: Promptable General Purpose Service Robot System with Foundation Models and Self-Recoverycited by 6
  25. 2024Consolidating Trees of Robotic Plans Generated Using Large Language Models to Improve Reliabilitycited by 6
  26. 2023Dobby: A Conversational Service Robot Driven by GPT-4cited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.