# Commonsense-Grounded Path Planning from Abstract Instructions

arXiv: https://arxiv.org/abs/2609.22813 · HTML: https://mlsanity.com/paper/2609.22813/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 34 references that are in the map.
- First posted on arXiv 19 Sep 2026.

## Builds on (most cited first, 26 of 26)

- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI](https://mlsanity.com/paper/2109.08238/) (2021)
- [LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action](https://mlsanity.com/paper/2207.04429/) (2022)
- [ProcTHOR: Large-Scale Embodied AI Using Procedural Generation](https://mlsanity.com/paper/2206.06994/) (2022)
- [Visual Language Maps for Robot Navigation](https://mlsanity.com/paper/2210.05714/) (2022)
- [Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting](https://mlsanity.com/paper/2306.17563/) (2023)
- [NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models](https://mlsanity.com/paper/2305.16986/) (2023)
- [NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Models](https://mlsanity.com/paper/2407.12366/) (2024)
- [Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planning](https://mlsanity.com/paper/2310.10103/) (2023)
- [MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigation](https://mlsanity.com/paper/2401.07314/) (2024)
- [LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Models](https://mlsanity.com/paper/2307.07889/) (2023)
- [VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Models](https://mlsanity.com/paper/2404.00210/) (2024)
- [Liability, Ethics, and Culture-Aware Behavior Specification using Rulebooks](https://mlsanity.com/paper/1902.09355/) (2019)
- [CoNVOI: Context-aware Navigation using Vision Language Models in Outdoor and Indoor Environments](https://mlsanity.com/paper/2403.15637/) (2024)
- [Safety Guardrails for LLM-Enabled Robots](https://mlsanity.com/paper/2503.07885/) (2025)
- [SocNavBench: A Grounded Simulation Testing Framework for Evaluating Social Navigation](https://mlsanity.com/paper/2103.00047/) (2021)
- [SocialNav: Training Human-Inspired Foundation Model for Socially-Aware Embodied Navigation](https://mlsanity.com/paper/2511.21135/) (2025)
- [BehAV: Behavioral Rule Guided Autonomy Using VLMs for Robot Navigation in Outdoor Scenes](https://mlsanity.com/paper/2409.16484/) (2024)
- [Guide-LLM: An Embodied LLM Agent and Text-Based Topological Map for Robotic Guidance of People with Visual Impairments](https://mlsanity.com/paper/2410.20666/) (2024)
- [Incremental Sampling-based Algorithm for Minimum-violation Motion Planning](https://mlsanity.com/paper/1305.1102/) (2013)
- [Minimum-violation LTL Planning with Conflicting Specifications](https://mlsanity.com/paper/1303.3679/) (2013)
- [From Obstacles to Etiquette: Robot Social Navigation with VLM-Informed Path Selection](https://mlsanity.com/paper/2602.09002/) (2026)
- [CANVAS: Commonsense-Aware Navigation System for Intuitive Human-Robot Interaction](https://mlsanity.com/paper/2410.01273/) (2024)
- [Path Planning using Instruction-Guided Probabilistic Roadmaps](https://mlsanity.com/paper/2502.16515/) (2025)
- [LISN: Language-Instructed Social Navigation with VLM-based Controller Modulating](https://mlsanity.com/paper/2512.09920/) (2025)
- [NORM-Nav: Zero-Shot Mobile Robot Navigation with Natural Language Behavioral Constraints](https://mlsanity.com/paper/2605.16979/) (2026)

## Abstract (arXiv)

We present commonsense ranked search (CoRS), a novel path planner that turns an abstract instruction into a route that follows commonsense. While existing methods respect the considerations written down in advance, a robot working among people must follow those left unstated too, as with a wet floor that a worker avoids without being told. CoRS leverages large language models (LLMs) and vision-language models (VLMs) as commonsense knowledge to reason about these latent considerations in its planning. Given an abstract instruction (e.g., ``move carefully'') and visual observations of each region in the environment, CoRS derives a consideration for each region, as in ``this wet floor is slippery and worth a detour.'' It then compares the considerations between regions to see which of the two the robot should avoid more, as in ``the crowd is worse than the wet floor.'' These judgments sort the regions into a commonsense ranking, whose costs drive a conventional search that always returns a valid route. We build a benchmark for planning under latent considerations, with three environments, 1350 problems, and five instructions at three levels of abstraction. Experiments show that CoRS discovers the unstated considerations and goes around the ones worth a detour while crossing the rest, a behavior that recent LLM-based planners do not achieve.