Learning to Tune Like an Expert: Interpretable and Scene-Aware Navigation via MLLM Reasoning and CVAE-Based Adaptation
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Service robots are increasingly deployed in diverse and dynamic environments, where both physical layouts and social contexts change over time and across locations. In these unstructured settings, conventional navigation systems that rely on fixed parameters often fail to generalize across scenarios, resulting in degraded performance and reduced social acceptance. Although recent approaches have leveraged reinforcement learning to enhance traditional planners, these methods often fail in real-world deployments due to poor generalization and limited simulation diversity, which hampers effective sim-to-real transfer. To tackle these issues, we present LE-Nav, an interpretable and scene-aware navigation framework that leverages multi-modal large language model reasoning and conditional variational autoencoders to adaptively tune planner hyperparameters. To achieve zero-shot scene understanding, we utilize one-shot exemplars and chain-of-thought prompting strategies. Additionally, a conditional variational autoencoder captures the mapping between natural language instructions and navigation hyperparameters, enabling expert-level tuning. Experiments show that LE-Nav can generate hyperparameters achieving human-level tuning across diverse planners and scenarios. Real-world navigation trials and a user study on a smart wheelchair platform demonstrate that it outperforms state-of-the-art methods on quantitative metrics such as success rate, efficiency, safety, and comfort, while receiving higher subjective scores for perceived safety and social acceptance. Code is available at https://github.com/Cavendish518/LE-Nav.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 24 of its 51 references that are in the map. First posted on arXiv 15 Jul 2025.
Builds on
24 of 51 references.
Closest ancestors · 1
Same direction, newest first
- 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning12 Mar
Other references · 23
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2024GPT-4o System Cardcited by 4,588
- 2023Qwen Technical Reportcited by 3,344
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
- 2022Planning-oriented Autonomous Drivingcited by 1,087
- 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 439
- 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 425
11 more
- 2025Kimi-VL Technical Reportcited by 388
- 2023GPT-Driver: Learning to Drive with GPTcited by 347
- 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 141
- 2023L3MVN: Leveraging Large Language Models for Visual Target Navigationcited by 120
- 2024Driving Everywhere with Large Language Model Policy Adaptationcited by 53
- 2023Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrationscited by 49
- 2024VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Modelscited by 46
- 2024A new sociology of humans and machinescited by 32
- 2024NeSLAM: Neural Implicit Mapping and Self-Supervised Feature Tracking With Depth Completion and Denoisingcited by 30
- 2020Dynamically Feasible Deep Reinforcement Learning Policy for Robot Navigation in Dense Mobile Crowdscited by 16
- 2025A Player Selection Network for Scalable Game-Theoretic Prediction and Planningcited by 1
Built on by
Citing papers per year: 2025: 4; 2026: 2.
6 papers.
- 2025GaussianDWM: 3D Gaussian Driving World Model for Unified Scene Understanding and Multi-Modal Generationcited by 23
- 2025UniPR-3D: Towards Universal Visual Place Recognition with Visual Geometry Grounded Transformercited by 8
- 2025Reloc-VGGT: Visual Re-localization with Geometry Grounded Transformercited by 5
- 2026CLOT: Closed-Loop Global Motion Tracking for Whole-Body Humanoid Teleoperationcited by 4
- 2025Guided Diffusion-based Generation of Adversarial Objects for Real-World Monocular Depth Estimation Attackscited by 1
- 2026GaussianDWM++: Language-Grounded 3D Gaussian Driving World Model for Unified Scene Understanding, Editing, and Multi-Modal Generationcited by 0
- 2026GaussianDWM++: Language-Grounded 3D Gaussian Driving World Model for Unified Scene Understanding, Editing, and Multi-Modal Generation17 Aug 2026
- 2026CLOT: Closed-Loop Global Motion Tracking for Whole-Body Humanoid Teleoperation13 Feb 2026
- 2025Guided Diffusion-based Generation of Adversarial Objects for Real-World Monocular Depth Estimation Attacks30 Dec 2025
- 2025GaussianDWM: 3D Gaussian Driving World Model for Unified Scene Understanding and Multi-Modal Generation29 Dec 2025
- 2025Reloc-VGGT: Visual Re-localization with Geometry Grounded Transformer26 Dec 2025
- 2025UniPR-3D: Towards Universal Visual Place Recognition with Visual Geometry Grounded Transformer24 Dec 2025
Similar, not cited
Close in wording, no citation either way
- 2025FiLM-Nav: Efficient and Generalizable Navigation via VLM Fine-tuning
- 2026APPLV: Adaptive Planner Parameter Learning from Vision-Language-Action Model
- 2026Interpreting Context-Aware Human Preferences for Multi-Objective Robot Navigation ↗
- 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models
- 2026SuperNav: An Agentic Navigation System for Any Task in Any Scene
- 2026Think When It Matters: Conditional VLM Reasoning for Social Navigation with RL Policies
- 2026NavGPT-3: Harnessing Context in a Hierarchical Navigation Runtime
- 2025NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environments