mlsanityData to 14 Oct 2026

Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigation

arXiv 2605.27582 ↗First posted 26 May 2026cs.RO cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Embodied navigation requires an agent to map language and visual observations to a stream of spatial actions that drive a real robot through environments it has never seen. The dominant approach has been to scale vision-language-action (VLA) foundation models on ever-larger collections of robot trajectories. This paper argues that, for navigation specifically, generality can be obtained structurally, not only through data scale. The underlying decision structure of navigation reduces to a single Language-Vision-Robot Actions Translation. The language action emits semantic-level directional command and the vision action emits a pixel-level visual target. Both outputs lie inside the natural output manifold of pretrained multimodal large language models (MLLMs), so the task can be reasoned about by an agent rather than learned from robot data. Therefore, we present Uni-LaViRA, a unified agentic architecture that extends the same insight to four task families (VLN-CE, ObjectNav, EQA, and Aerial-VLN) and to four heterogeneous real robots (Wheeled, Quadruped, Humanoid robot, and a self-built UAV) in a zero-shot manner. Two agent-loop mechanisms make this unification practical. TODO List Memory (TDM) rewrites a structured checklist of pending sub-goals at every step, reciting the unfinished items back into the agent's most recent attention window. Second Chance Backtrack (SCB) rolls the robot back to the pre-error state and conditions the agent's next plan on the failed sub-trajectory, turning single-pass navigation into a self-correcting process. With zero training effort, Uni-LaViRA reaches 60.7% SR on VLN-CE R2R, 51.3% on VLN-CE RxR, 77.7% on HM3D-v2, 60.0% on HM3D-OVON, 54.7% on MP3D-EQA, and 40.0% on OpenUAV, matching or even surpassing recent training navigation foundation models that consume millions of samples and thousands of GPU-hours.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 71 of its 78 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

71 of 78 references.

References · 71

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Segment Anythingcited by 8,234
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  7. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  9. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  10. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  11. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  12. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
38 more
  1. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  2. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  3. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  4. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 556
  5. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  6. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  7. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 362
  8. 2017Embodied Question Answeringcited by 336
  9. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
  10. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  11. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 244
  12. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
  13. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 211
  14. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 205
  15. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
  16. 2024Agent S: An Open Agentic Framework that Uses Computers Like a Humancited by 187
  17. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
  18. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 157
  19. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 156
  20. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 138
  21. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 127
  22. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
  23. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 122
  24. 2023L3MVN: Leveraging Large Language Models for Visual Target Navigationcited by 120
  25. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 118
  26. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 117
  27. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
  28. 2022Habitat-Matterport 3D Semantics Datasetcited by 110
  29. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 110
  30. 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 109
  31. 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 95
  32. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 94
  33. 2023PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNavcited by 93
  34. 2023GOAT: GO to Any Thingcited by 89
  35. 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 89
  36. 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 86
  37. 2024HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigationcited by 71
  38. 2023OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNavcited by 70

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026HAM-VLN: Harnessing Hierarchical Agentic Memory for Zero-Shot Vision-and-Language Navigationcited by 2
  2. 2026Air-Ground Collaborative Vision-and-Language Navigation via Shared Bird's-Eye Mapscited by 1
  3. 2026RoboFind: Multi-Agent Personalized Object Search for People Who Are Blind or Have Low Visioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Vision-and-Language Navigation for UAVs: Progress, Challenges, and a Research Roadmap
  2. 2026GemNav: Discrete-Token Visual Robot Navigation using a Multimodal Large Language Model
  3. 2025A Navigation Framework Utilizing Vision-Language Models
  4. 2026AgentVLN: Towards Agentic Vision-and-Language Navigation
  5. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
  6. 2026Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines
  7. 2025UAV-VLN: End-to-End Vision Language guided Navigation for UAVs
  8. 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges