# Planning in the Dark: LLM-Symbolic Planning Pipeline without Experts

arXiv: https://arxiv.org/abs/2409.15915 · HTML: https://mlsanity.com/paper/2409.15915/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 70 references that are in the map.
- First posted on arXiv 24 Sep 2024.

## Builds on (most cited first, 16 of 16)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](https://mlsanity.com/paper/2304.11477/) (2023)
- [On the Planning Abilities of Large Language Models : A Critical Investigation](https://mlsanity.com/paper/2305.15771/) (2023)
- [Large Language Models are Zero-Shot Rankers for Recommender Systems](https://mlsanity.com/paper/2305.08845/) (2023)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning](https://mlsanity.com/paper/2305.14909/) (2023)
- [Language Models Meet World Models: Embodied Experiences Enhance Language Models](https://mlsanity.com/paper/2305.10626/) (2023)
- [PDDLGym: Gym Environments from PDDL Problems](https://mlsanity.com/paper/2002.06432/) (2020)
- [Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking](https://mlsanity.com/paper/2310.13243/) (2023)
- [On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)](https://mlsanity.com/paper/2401.02500/) (2024)
- [ChatGLM-RLHF: Practices of Aligning Large Language Models with Human Feedback](https://mlsanity.com/paper/2404.00934/) (2024)

## Built on by (most cited first, 16 of 16)

- [LLMs as Planning Formalizers: A Survey for Leveraging Large Language Models to Construct Automated Planning Models](https://mlsanity.com/paper/2503.18971/) (2025)
- [Explain Before You Answer: A Survey on Compositional Visual Reasoning](https://mlsanity.com/paper/2508.17298/) (2025)
- [Teaching LLMs to Plan: Logical Chain-of-Thought Instruction Tuning for Symbolic Planning](https://mlsanity.com/paper/2509.13351/) (2025)
- [Exploring Consciousness in LLMs: A Systematic Survey of Theories, Implementations, and Frontier Risks](https://mlsanity.com/paper/2505.19806/) (2025)
- [Symmetry-Aware Transformer Training for Automated Planning](https://mlsanity.com/paper/2508.07743/) (2025)
- [TAPE: Tool-Guided Adaptive Planning and Constrained Execution in Language Model Agents](https://mlsanity.com/paper/2602.19633/) (2026)
- [Agentic LLM Planning via Step-Wise PDDL Simulation: An Empirical Characterisation](https://mlsanity.com/paper/2603.06064/) (2026)
- [Model Space Reasoning as Search in Feedback Space for Planning Domain Generation](https://mlsanity.com/paper/2604.08712/) (2026)
- [Planning in the LLM Era: Building for Reliability and Efficiency](https://mlsanity.com/paper/2605.21902/) (2026)
- [What We Talk About When We Talk About LLM Planning: Evidence for Two Distinct Planning Abilities](https://mlsanity.com/paper/2607.11197/) (2026)
- [Scaling Multi-Agent Epistemic Planning through GNN-Derived Heuristics](https://mlsanity.com/paper/2508.12840/) (2025)
- [Generative AI-assisted Participatory Modeling in Socio-Environmental Planning under Deep Uncertainty](https://mlsanity.com/paper/2603.17021/) (2026)
- [Toward Secure and Reliable PDDL Formalization of Large Language Models with Planner-in-the-Loop Feedback](https://mlsanity.com/paper/2606.29700/) (2026)
- [PDDLCoder: Agentic PDDL Generation for LLM-Assisted Symbolic Planning](https://mlsanity.com/paper/2608.16637/) (2026)
- [LLM-Only PDDL Domain Repair with Open-Weight Models](https://mlsanity.com/paper/2608.17341/) (2026)
- [CVE2AP: Automated Generation of PDDL-Encoded Attack Paths via Large Language Models](https://mlsanity.com/paper/2610.03383/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have shown promise in solving natural language-described planning tasks, but their direct use often leads to inconsistent reasoning and hallucination. While hybrid LLM-symbolic planning pipelines have emerged as a more robust alternative, they typically require extensive expert intervention to refine and validate generated action schemas. It not only limits scalability but also introduces a potential for biased interpretation, as a single expert's interpretation of ambiguous natural language descriptions might not align with the user's actual intent. To address this, we propose a novel approach that constructs an action schema library to generate multiple candidates, accounting for the diverse possible interpretations of natural language descriptions. We further introduce a semantic validation and ranking module that automatically filter and rank the generated schemas and plans without expert-in-the-loop. The experiments showed our pipeline maintains superiority in planning over the direct LLM planning approach. These findings demonstrate the feasibility of a fully automated end-to-end LLM-symbolic planner that requires no expert intervention, opening up the possibility for a broader audience to engage with AI planning with less prerequisite of domain expertise.