# Safety Aware Task Planning via Large Language Models in Robotics

arXiv: https://arxiv.org/abs/2503.15707 · HTML: https://mlsanity.com/paper/2503.15707/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 19 Mar 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 28 of 28)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought](https://mlsanity.com/paper/2305.15021/) (2023)
- [Building Cooperative Embodied Agents Modularly with Large Language Models](https://mlsanity.com/paper/2307.02485/) (2023)
- [Text2Motion: From Natural Language Instructions to Feasible Plans](https://mlsanity.com/paper/2303.12153/) (2023)
- [LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving](https://mlsanity.com/paper/2310.03026/) (2023)
- [SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models](https://mlsanity.com/paper/2309.10062/) (2023)
- [Large Language Models for Robotics: A Survey](https://mlsanity.com/paper/2311.07226/) (2023)
- [AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation](https://mlsanity.com/paper/2410.00371/) (2024)
- [SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents](https://mlsanity.com/paper/2412.13178/) (2024)
- [Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement](https://mlsanity.com/paper/2402.11436/) (2024)
- [Discuss Before Moving: Visual Language Navigation via Multi-expert Discussions](https://mlsanity.com/paper/2309.11382/) (2023)
- [Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agents](https://mlsanity.com/paper/2309.09919/) (2023)
- [COHERENT: Collaboration of Heterogeneous Multi-Robot System with Large Language Models](https://mlsanity.com/paper/2409.15146/) (2024)
- [CodeJudge: Evaluating Code Generation with Large Language Models](https://mlsanity.com/paper/2410.02184/) (2024)
- [Interactive Planning Using Large Language Models for Partially Observable Robotics Tasks](https://mlsanity.com/paper/2312.06876/) (2023)
- [Comparative Analysis of Control Barrier Functions and Artificial Potential Fields for Obstacle Avoidance](https://mlsanity.com/paper/2010.09819/) (2020)
- [PhysicsAssistant: An LLM-Powered Interactive Learning Robot for Physics Lab Investigations](https://mlsanity.com/paper/2403.18721/) (2024)
- [Safety Control of Service Robots with LLMs and Embodied Knowledge Graphs](https://mlsanity.com/paper/2405.17846/) (2024)
- [Safe Certificate-Based Maneuvers for Teams of Quadrotors Using Differential Flatness](https://mlsanity.com/paper/1702.01075/) (2017)
- [GRACE: Generating Socially Appropriate Robot Actions Leveraging LLMs and Human Explanations](https://mlsanity.com/paper/2409.16879/) (2024)

## Built on by (most cited first, 24 of 24)

- [Can AI Perceive Physical Danger and Intervene?](https://mlsanity.com/paper/2509.21651/) (2025)
- [AEROS: A Single-Agent Operating Architecture with Embodied Capability Modules](https://mlsanity.com/paper/2604.07039/) (2026)
- [MADRA: Multi-Agent Debate for Risk-Aware Embodied Planning](https://mlsanity.com/paper/2511.21460/) (2025)
- [Hallucination-Aware Generative Pretrained Transformer for Cooperative Aerial Mobility Control](https://mlsanity.com/paper/2504.10831/) (2025)
- [Compositional Coordination for Multi-Robot Teams with Large Language Models](https://mlsanity.com/paper/2507.16068/) (2025)
- [EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agents](https://mlsanity.com/paper/2605.30924/) (2026)
- [Harness Engineering for Physical AI: Robot Middleware Is the Harness Layer](https://mlsanity.com/paper/2606.09416/) (2026)
- [LVLM-MPC Collaboration for Autonomous Driving: A Safety-Aware and Task-Scalable Control Architecture](https://mlsanity.com/paper/2505.04980/) (2025)
- [RoboPARA: Dual-Arm Robot Planning with Parallel Allocation and Recomposition Across Tasks](https://mlsanity.com/paper/2506.06683/) (2025)
- [Grounding Language Models with Semantic Digital Twins for Robotic Planning](https://mlsanity.com/paper/2506.16493/) (2025)
- [HyCodePolicy: Hybrid Language Controllers for Multimodal Monitoring and Decision in Embodied Agents](https://mlsanity.com/paper/2508.02629/) (2025)
- [ResponsibleRobotBench: Benchmarking Responsible Robot Manipulation using Multi-modal Large Language Models](https://mlsanity.com/paper/2512.04308/) (2025)
- [LabGuard: Grounding Natural-Language Laboratory Rules into Runtime Guards for Embodied Laboratory Agents](https://mlsanity.com/paper/2606.31045/) (2026)
- [Semantic Intelligence: Integrating GPT-4 with A Planning in Low-Cost Robotics](https://mlsanity.com/paper/2505.01931/) (2025)
- [Understanding Physical Properties of Unseen Deformable Objects by Leveraging Large Language Models and Robot Actions](https://mlsanity.com/paper/2506.03760/) (2025)
- [Human Centric General Physical Intelligence for Agile Manufacturing Automation](https://mlsanity.com/paper/2508.11960/) (2025)
- [A Hazard-Informed Data Pipeline for Robotics Physical Safety](https://mlsanity.com/paper/2603.06130/) (2026)
- [Position: A Three-Layer Probabilistic Assume-Guarantee Architecture Is Structurally Required for Safe LLM Agent Deployment](https://mlsanity.com/paper/2605.18672/) (2026)
- [Agentic Harnesses: LLM-Driven Verification Layers for Robot Autonomy](https://mlsanity.com/paper/2608.09857/) (2026)
- [MaCoPlanner: LLM-Assisted Manual-Compiled Task Planning with Proactive Safety Verification for Robotic Industrial Panel Operation](https://mlsanity.com/paper/2608.28300/) (2026)
- [PanelShield: Verifiable Closed-Loop Safe Planning for Robotic Industrial Panel Operation](https://mlsanity.com/paper/2608.28305/) (2026)
- [Safe Task Planning with Long-Term Graph Memory for Embodied Agents](https://mlsanity.com/paper/2609.08444/) (2026)
- [Large Language Models in the Loop: A Stability- and Network-Aware Survey in Networked Control, Cyber-Physical, and Multi-Agent Systems](https://mlsanity.com/paper/2609.16599/) (2026)
- [Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directions](https://mlsanity.com/paper/2609.27847/) (2026)

## Abstract (arXiv)

The integration of large language models (LLMs) into robotic task planning has unlocked better reasoning capabilities for complex, long-horizon workflows. However, ensuring safety in LLM-driven plans remains a critical challenge, as these models often prioritize task completion over risk mitigation. This paper introduces SAFER (Safety-Aware Framework for Execution in Robotics), a multi-LLM framework designed to embed safety awareness into robotic task planning. SAFER employs a Safety Agent that operates alongside the primary task planner, providing safety feedback. Additionally, we introduce LLM-as-a-Judge, a novel metric leveraging LLMs as evaluators to quantify safety violations within generated task plans. Our framework integrates safety feedback at multiple stages of execution, enabling real-time risk assessment, proactive error correction, and transparent safety evaluation. We also integrate a control framework using Control Barrier Functions (CBFs) to ensure safety guarantees within SAFER's task planning. We evaluated SAFER against state-of-the-art LLM planners on complex long-horizon tasks involving heterogeneous robotic agents, demonstrating its effectiveness in reducing safety violations while maintaining task efficiency. We also verify the task planner and safety planner through actual hardware experiments involving multiple robots and a human.