Structured World-State Reasoning for Agentic Robotic Search
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Long-horizon robotic search must resolve natural language against heterogeneous, incomplete, and often ambiguous evidence: textual information, prior maps, and observations arriving over time. The core challenge is to contextualize these streams and decide where to gather evidence before selecting a target. We present WORLDS: World-state Observation and Reasoning for Language-guided Discovery and Search, a framework that grounds reasoning in a persistent graph initialized from geospatial priors and updated by perception. Parallel Reasoners maintain competing candidate interpretations and request evidence to distinguish between them. We collect and process the requested observations with a multimodal Examiner, after which a Judge selects a grounded target or requests another pass. WORLDS achieves 51.8% navigation success across all 5,311 CityNav test episodes, the highest reported success rate, exceeding the previous published best by 15.7 percentage points under an OSM-only, high-resolution orthographic protocol. On 1,000 shared episodes, it achieves 50.0% versus 27.9% for the strongest adapted baseline using the same model, prior, sensing stack, and movement budget. Observation-based verification by the Examiner contributes 5.9 points of this success, and at a reduced reasoning-effort setting WORLDS still exceeds the adapted GeoNav baseline by 18.8 points while generating fewer tokens. We also demonstrate WORLDS on a quadrotor, which flies the generated sensing waypoints and grounds three language targets, including a vehicle absent from the map, from its onboard imagery.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 32 of its 39 references that are in the map. First posted on arXiv 20 Sep 2026.
Builds on
32 of 39 references.
References · 32
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
- 2018On Evaluation of Embodied Navigation Agentscited by 526
- 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
- 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
- 2022Visual Language Maps for Robot Navigationcited by 362
- 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
- 2017Embodied Question Answeringcited by 335
20 more
- 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 298
- 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 291
- 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
- 2022Open-vocabulary Queryable Scene Representations for Real World Planningcited by 159
- 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 117
- 2020Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challengescited by 80
- 2025Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifierscited by 58
- 2018Interactive Visual Grounding of Referring Expressions for Human-Robot Interactioncited by 55
- 2022Aerial Vision-and-Dialog Navigationcited by 48
- 2024CityNav: A Large-Scale Dataset for Real-World Aerial Navigationcited by 40
- 2025FlightGPT: Towards Generalizable and Interpretable UAV Vision-and-Language Navigation with Vision-Language Modelscited by 22
- 2024Exploring Spatial Representation to Enhance LLM Reasoning in Aerial Vision-Language Navigationcited by 18
- 2025GeoNav: Empowering MLLMs with dual-scale geospatial reasoning for language-goal aerial navigationcited by 16
- 2023CityRefer: Geography-aware 3D Visual Grounding Dataset on City-scale Point Cloud Datacited by 15
- 2025RAVEN: Resilient Aerial Navigation via Open-Set Semantic Memory and Behavior Adaptationcited by 10
- 2020Spatial Language Understanding for Object Search in Partially Observed City-scale Environmentscited by 7
- 2026AirHunt: Bridging VLM Semantics and Continuous Planning for Efficient Aerial Object Navigationcited by 4
- 2026ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigationcited by 2
- 2026HTNav: A Hybrid Navigation Framework with Tiered Structure for Urban Aerial Vision-and-Language Navigationcited by 2
- 2026CLOSER-VLN: Closed-Loop Self-Verified Retrieval-Augmented Reasoning for Aerial Vision-Language Navigationcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025GET: Goal-directed Exploration and Targeting for Large-Scale Unknown Environments
- 2026Scaffolding Foundation Models into Physical-World Agents Pushes the Frontier of Long-Horizon Navigation
- 2026Ask the World Before Acting: Environment Probing for Calibrated Agent World Models
- 2026HAM-VLN: Harnessing Hierarchical Agentic Memory for Zero-Shot Vision-and-Language Navigation
- 2026Semantic Area Graph Reasoning for Multi-Robot Language-Guided Search
- 2026STaR: Scalable Task-Conditioned Retrieval for Long-Horizon Multimodal Robot Memory
- 2026Hypothesis Graph Refinement: Hypothesis-Driven Exploration with Cascade Error Correction for Embodied Navigation
- 2025Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Models