PlanGuard: A Guardrail for Multi-Step Plan Safety in Embodied Agents
On the map: LLM safety and security → Prompt injection and agent security also fits: Safety alignment, refusal and guardrails
Abstract (arXiv)
Embodied task planners may produce multi-step plans whose subtask dependencies and interactions with the environment create physical risks during execution. Yet existing safeguards overlook such compositional risks, as general-purpose guardrails focus on semantic harm and embodied safety detectors assess subtasks in isolation. To address this gap, we introduce PlanGuard, the first pre-execution detector that evaluates the physical safety of a complete multi-step plan in its current environment. For training and evaluation, we construct a Multi-Step Plan Safety (MSP-Safe) dataset through paired task construction, plan generation using diverse planners, and safety annotation by three judges. Task-oriented SFT on MSP-Safe establishes fundamental plan-safety assessment capabilities, yet a substantial gap remains between compact models suitable for real-time deployment and stronger but costlier large models. Accordingly, we propose Strong-Teacher Adaptive Compensation for On-Policy Distillation (STAC-OPD), which provides compact models with adaptive strong-teacher supervision along their on-policy trajectories. It combines token-level distribution transfer from a fine-tuned strong teacher with probability-routed sequence-level compensation, retaining student-generated targets when the student favors the reference safety decision and using teacher-reconstructed targets otherwise. Across all test subsets, PlanGuard-2B achieves average 87.15% ACC and 87.21% F1, demonstrating effective whole-plan physical-risk detection at compact model scale. Code and dataset will be publicly released.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 19 of its 24 references that are in the map. First posted on arXiv 26 Sep 2026.
Builds on
19 of 24 references.
Closest ancestors · 1
Same direction, newest first
Other references · 18
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
- 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
- 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
- 2025Qwen3Guard Technical Reportcited by 155
- 2024SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agentscited by 116
- 2024Manipulate-Anything: Automating Real-World Robots using Vision-Language Modelscited by 83
- 2025Igniting VLMs toward the Embodied Spacecited by 53
- 2024LoTa-Bench: Benchmarking Language-oriented Task Planners for Embodied Agentscited by 51
- 2026Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Executioncited by 45
- 2025Generating Robot Constitutions & Benchmarks for Semantic Safetycited by 33
6 more
- 2025A Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agentscited by 30
- 2024EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agentscited by 27
- 2025Subtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision Makingcited by 20
- 2025SafeMind: Benchmarking and Mitigating Safety Risks in Embodied LLM Agentscited by 17
- 2025Can AI Perceive Physical Danger and Intervene?cited by 14
- 2025GuardTrace-VL: Detecting Unsafe Multimodel Reasoning via Iterative Safety Supervisioncited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Using large language models for embodied planning introduces systematic safety risks
- 2025RoboSafe: Safeguarding Embodied Agents via Executable Safety Logic
- 2025MADRA: Multi-Agent Debate for Risk-Aware Embodied Planning
- 2025Building a Foundational Guardrail for General Agentic Systems via Synthetic Data
- 2026Self-Evolving Just-In-Time Memory for Proactive Embodied Safety
- 2025IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Tasks
- 2026SafeBranch: Branch-Pair Safety Alignment for Embodied Agents
- 2025SENTINEL: A Multi-Level Formal Framework for Safety Evaluation of Foundation Model-based Embodied Agents