LLM-Guided Agentic Floor Plan Parsing for Accessible Indoor Navigation of Blind and Low-Vision People
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Indoor navigation remains a critical accessibility challenge for the blind and low-vision (BLV) individuals, as existing solutions rely on costly per-building infrastructure. We present an agentic framework that converts a single floor plan image into a structured, retrievable knowledge base to generate safe, accessible navigation instructions with lightweight infrastructure. The system has two phases: a multi-agent module that parses the floor plan into a spatial knowledge graph through a self-correcting pipeline with iterative retry loops and corrective feedback; and a Path Planner that generates accessible navigation instructions, with a Safety Evaluator agent assessing potential hazards along each route. We evaluate the system on the real-world UMBC Math and Psychology building (floors MP-1 and MP-3) and on the CVC-FP benchmark. On MP-1, we achieve success rates of 92.31%, 76.92%, and 61.54% for short, medium, and long routes, outperforming the strongest single-call baseline (Claude 3.7 Sonnet) at 84.62%, 69.23%, and 53.85%. On MP-3, we reach 76.92%, 61.54%, and 38.46%, compared to the best baseline at 61.54%, 46.15%, and 23.08%. These results show consistent gains over single-call LLM baselines and demonstrate that our workflow is a scalable solution for accessible indoor navigation for BLV individuals.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 23 of its 33 references that are in the map. First posted on arXiv 27 Apr 2026.
Builds on
23 of 33 references.
References · 23
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
- 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
- 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
- 2025YOLOv12: Attention-Centric Real-Time Object Detectorscited by 300
- 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
11 more
- 2020PP-OCR: A Practical Ultra Lightweight OCR Systemcited by 147
- 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 85
- 2017A Survey of Indoor Localization Systems and Technologiescited by 77
- 2022HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoisingcited by 72
- 2021FloorPlanCAD: A Large-Scale CAD Drawing Dataset for Panoptic Symbol Spottingcited by 28
- 2024Scalable and Accurate Graph Reasoning with LLM-based Multi-Agentscited by 21
- 2024Vision Language Models Can Parse Floor Plan Mapscited by 5
- 2024WAFFLE: Multimodal Floorplan Understanding in the Wildcited by 5
- 2025LLM-Guided Indoor Navigation with Multimodal Map Understandingcited by 4
- 2021ResVGAE: Going Deeper with Residual Modules for Link Predictioncited by 2
- 2025Floorplan2Guide: LLM-Guided Floorplan Parsing for BLV Indoor Navigationcited by 2
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Towards an automated AI-based framework for floor plan compliance checks for residential buildingscited by 0
- 2026When Should a Network Emit Geometry, and When Should It Detect It? Readout, Reconciliation, and Representation in Floorplan Vectorizationcited by 0
- 2026Beyond Scene Description: Multi-Agent Orchestration for Non-visual Access to Virtual Worldscited by 0
- 2026Beyond Scene Description: Multi-Agent Orchestration for Non-visual Access to Virtual Worlds13 Sep 2026
- 2026When Should a Network Emit Geometry, and When Should It Detect It? Readout, Reconciliation, and Representation in Floorplan Vectorization26 Aug 2026
- 2026Towards an automated AI-based framework for floor plan compliance checks for residential buildings26 May 2026
Similar, not cited
Close in wording, no citation either way
- 2024Guide-LLM: An Embodied LLM Agent and Text-Based Topological Map for Robotic Guidance of People with Visual Impairments
- 2026CapNav: Benchmarking Vision Language Models on Capability-conditioned Indoor Navigation
- 2025Fine-Tuning Vision-Language Models for Visual Navigation Assistance
- 2026Position: Assistive Agents Need Accessibility Alignment
- 2025OpenGuide: Assistive Object Retrieval in Indoor Spaces for Individuals with Visual Impairments ↗
- 2026Bespoke Visual Assistance: What and How do Blind and Low-Vision People Create with Agentic Programming?
- 2021Pathdreamer: A World Model for Indoor Navigation
- 2025Efficient Navigation in Unknown Indoor Environments with Vision-Language Models