# SELP: Generating Safe and Efficient Task Plans for Robot Agents with Large Language Models

arXiv: https://arxiv.org/abs/2409.19471 · HTML: https://mlsanity.com/paper/2409.19471/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 32 of its 37 references that are in the map.
- First posted on arXiv 28 Sep 2024.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 32)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Inner Monologue: Embodied Reasoning through Planning with Language Models](https://mlsanity.com/paper/2207.05608/) (2022)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [ProgPrompt: Generating Situated Robot Task Plans using Large Language Models](https://mlsanity.com/paper/2209.11302/) (2022)
- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](https://mlsanity.com/paper/2304.11477/) (2023)
- [LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action](https://mlsanity.com/paper/2207.04429/) (2022)
- [VIMA: General Robot Manipulation with Multimodal Prompts](https://mlsanity.com/paper/2210.03094/) (2022)
- [ChatGPT for Robotics: Design Principles and Model Abilities](https://mlsanity.com/paper/2306.17582/) (2023)
- [TidyBot: Personalized Robot Assistance with Large Language Models](https://mlsanity.com/paper/2305.05658/) (2023)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [LLM4Drive: A Survey of Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.01043/) (2023)
- [Translating Natural Language to Planning Goals with Large-Language Models](https://mlsanity.com/paper/2302.05128/) (2023)
- [AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkers](https://mlsanity.com/paper/2306.06531/) (2023)
- [Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control](https://mlsanity.com/paper/2103.02142/) (2021)
- [Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specifications](https://mlsanity.com/paper/1807.06333/) (2018)
- [Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agents](https://mlsanity.com/paper/2309.09919/) (2023)
- [KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents](https://mlsanity.com/paper/2403.03101/) (2024)
- [NL2TL: Transforming Natural Languages to Temporal Logics using Large Language Models](https://mlsanity.com/paper/2305.07766/) (2023)
- [Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments](https://mlsanity.com/paper/2302.11649/) (2023)
- [SayCanPay: Heuristic Planning with Large Language Models using Learnable Domain Knowledge](https://mlsanity.com/paper/2308.12682/) (2023)
- [Data-Efficient Learning of Natural Language to Linear Temporal Logic Translators for Robot Task Specification](https://mlsanity.com/paper/2303.08006/) (2023)
- [Learning a natural-language to LTL executable semantic parser for grounded robotics](https://mlsanity.com/paper/2008.03277/) (2020)
- [LLMs for Relational Reasoning: How Far are We?](https://mlsanity.com/paper/2401.09042/) (2024)
- [LLM-Based Human-Robot Collaboration Framework for Manipulation Tasks](https://mlsanity.com/paper/2308.14972/) (2023)

## Built on by (most cited first, 21 of 21)

- [VeriSafe Agent: Safeguarding Mobile GUI Agent via Logic-based Action Verification](https://mlsanity.com/paper/2503.18492/) (2025)
- [Towards Embodied Agentic AI: Review and Classification of LLM- and VLM-Driven Robot Autonomy and Interaction](https://mlsanity.com/paper/2508.05294/) (2025)
- [SafePlan: Leveraging Formal Logic and Chain-of-Thought Reasoning for Enhanced Safety in LLM-based Robotic Task Planning](https://mlsanity.com/paper/2503.06892/) (2025)
- [Multimodal Large Language Models-Enabled UAV Swarm: Towards Efficient and Intelligent Autonomous Aerial Systems](https://mlsanity.com/paper/2506.12710/) (2025)
- [A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data](https://mlsanity.com/paper/2601.17717/) (2026)
- [Constrained Decoding for Safe Robot Navigation Foundation Models](https://mlsanity.com/paper/2509.01728/) (2025)
- [Graphormer-Guided Task Planning: Beyond Static Rules with LLM Safety Perception](https://mlsanity.com/paper/2503.06866/) (2025)
- [Safe and Interpretable Multimodal Path Planning for Multi-Agent Cooperation](https://mlsanity.com/paper/2602.19304/) (2026)
- [Understanding Physical Properties of Unseen Deformable Objects by Leveraging Large Language Models and Robot Actions](https://mlsanity.com/paper/2506.03760/) (2025)
- [ROSClaw: A Hierarchical Semantic-Physical Framework for Heterogeneous Multi-Agent Collaboration](https://mlsanity.com/paper/2604.04664/) (2026)
- [Learning of Robot Safety Policies via Adversarial Synthetic Scenarios](https://mlsanity.com/paper/2606.05952/) (2026)
- [When Replanning Becomes the Bottleneck: Budgeted Replanning for Embodied Agents](https://mlsanity.com/paper/2608.01428/) (2026)
- [Specification-Aware Distribution Shaping for Robotics Foundation Models](https://mlsanity.com/paper/2603.17969/) (2026)
- [Object Search in Partially-Known Environments via LLM-informed Model-based Planning and Prompt Selection](https://mlsanity.com/paper/2603.23800/) (2026)
- [NEXUS: Continual Learning of Symbolic Constraints for Safe and Robust Embodied Planning](https://mlsanity.com/paper/2605.09387/) (2026)
- [HEART: Coordination of Heterogeneous Expert Agents for Physically Grounded Robotic Task Planning](https://mlsanity.com/paper/2606.25404/) (2026)
- [ViTL: Temporal Logic-Guided Zero-Shot Natural Language Navigation via Vision-Language Models](https://mlsanity.com/paper/2606.30696/) (2026)
- [SPINE: Bridging the Cyber-Physical Gap with Agentic AI](https://mlsanity.com/paper/2607.13049/) (2026)
- [MaCoPlanner: LLM-Assisted Manual-Compiled Task Planning with Proactive Safety Verification for Robotic Industrial Panel Operation](https://mlsanity.com/paper/2608.28300/) (2026)
- [PanelShield: Verifiable Closed-Loop Safe Planning for Robotic Industrial Panel Operation](https://mlsanity.com/paper/2608.28305/) (2026)
- [EmbodiedMind: Adaptive Data Curation and Prefix-Tree Reinforcement Learning for Efficient Embodied Intelligence](https://mlsanity.com/paper/2609.19659/) (2026)

## Abstract (arXiv)

Despite significant advancements in large language models (LLMs) that enhance robot agents' understanding and execution of natural language (NL) commands, ensuring the agents adhere to user-specified constraints remains challenging, particularly for complex commands and long-horizon tasks. To address this challenge, we present three key insights, equivalence voting, constrained decoding, and domain-specific fine-tuning, which significantly enhance LLM planners' capability in handling complex tasks. Equivalence voting ensures consistency by generating and sampling multiple Linear Temporal Logic (LTL) formulas from NL commands, grouping equivalent LTL formulas, and selecting the majority group of formulas as the final LTL formula. Constrained decoding then uses the generated LTL formula to enforce the autoregressive inference of plans, ensuring the generated plans conform to the LTL. Domain-specific fine-tuning customizes LLMs to produce safe and efficient plans within specific task domains. Our approach, Safe Efficient LLM Planner (SELP), combines these insights to create LLM planners to generate plans adhering to user commands with high confidence. We demonstrate the effectiveness and generalizability of SELP across different robot agents and tasks, including drone navigation and robot manipulation. For drone navigation tasks, SELP outperforms state-of-the-art planners by 10.8% in safety rate (i.e., finishing tasks conforming to NL commands) and by 19.8% in plan efficiency. For robot manipulation tasks, SELP achieves 20.4% improvement in safety rate. Our datasets for evaluating NL-to-LTL and robot task planning will be released in github.com/lt-asset/selp.