# Robot Task Planning and Situation Handling in Open Worlds

arXiv: https://arxiv.org/abs/2210.01287 · HTML: https://mlsanity.com/paper/2210.01287/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 4 Oct 2022.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Inner Monologue: Embodied Reasoning through Planning with Language Models](https://mlsanity.com/paper/2207.05608/) (2022)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [VirtualHome: Simulating Household Activities via Programs](https://mlsanity.com/paper/1806.07011/) (2018)
- [Symbolic Knowledge Distillation: from General Language Models to Commonsense Models](https://mlsanity.com/paper/2110.07178/) (2021)
- [Pre-Trained Language Models for Interactive Decision-Making](https://mlsanity.com/paper/2202.01771/) (2022)
- [Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2108.13161/) (2021)
- [Can Generative Pre-trained Language Models Serve as Knowledge Bases for Closed-book QA?](https://mlsanity.com/paper/2106.01561/) (2021)
- [Online Replanning in Belief Space for Partially Observable Task and Motion Problems](https://mlsanity.com/paper/1911.04577/) (2019)
- [Housekeep: Tidying Virtual Households using Commonsense Reasoning](https://mlsanity.com/paper/2205.10712/) (2022)
- [Task Planning in Robotics: an Empirical Comparison of PDDL-based and ASP-based Systems](https://mlsanity.com/paper/1804.08229/) (2018)
- [Leveraging Post Hoc Context for Faster Learning in Bandit Settings with Applications in Robot-Assisted Feeding](https://mlsanity.com/paper/2011.02604/) (2020)
- [Augmenting Knowledge through Statistical, Goal-oriented Human-Robot Dialog](https://mlsanity.com/paper/1907.03390/) (2019)

## Built on by (most cited first, 10 of 10)

- [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](https://mlsanity.com/paper/2304.11477/) (2023)
- [GPT-4V(ision) for Robotics: Multimodal Task Planning from Human Demonstration](https://mlsanity.com/paper/2311.12015/) (2023)
- [Benchmarks for Automated Commonsense Reasoning: A Survey](https://mlsanity.com/paper/2302.04752/) (2023)
- [Grounding Classical Task Planners via Vision-Language Models](https://mlsanity.com/paper/2304.08587/) (2023)
- [Towards Open-World Grasping with Large Vision-Language Models](https://mlsanity.com/paper/2406.18722/) (2024)
- [NovelGym: A Flexible Ecosystem for Hybrid Planning and Learning Agents Designed for Open Worlds](https://mlsanity.com/paper/2401.03546/) (2024)
- [AdaptBot: Combining LLM with Knowledge Graphs and Human Input for Generic-to-Specific Task Decomposition and Knowledge Refinement](https://mlsanity.com/paper/2502.02067/) (2025)
- [A Prompt-driven Task Planning Method for Multi-drones based on Large Language Model](https://mlsanity.com/paper/2406.00006/) (2024)
- [Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills](https://mlsanity.com/paper/2608.01851/) (2026)
- [Bridging Learned Visual Perception and Symbolic Belief-Space Planning](https://mlsanity.com/paper/2609.16884/) (2026)

## Abstract (arXiv)

Automated task planning algorithms have been developed to help robots complete complex tasks that require multiple actions. Most of those algorithms have been developed for "closed worlds" assuming complete world knowledge is provided. However, the real world is generally open, and the robots frequently encounter unforeseen situations that can potentially break the planner's completeness. This paper introduces a novel algorithm (COWP) for open-world task planning and situation handling that dynamically augments the robot's action knowledge with task-oriented common sense. In particular, common sense is extracted from Large Language Models based on the current task at hand and robot skills. For systematic evaluations, we collected a dataset that includes 561 execution-time situations in a dining domain, where each situation corresponds to a state instance of a robot being potentially unable to complete a task using a solution that normally works. Experimental results show that our approach significantly outperforms competitive baselines from the literature in the success rate of service tasks. Additionally, we have demonstrated COWP using a mobile manipulator. The project website is available at: https://cowplanning.github.io/, where a more detailed version can also be found. This version has been accepted for publication in Autonomous Robots.