mlsanityData to 7 Oct 2026

Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Tasks

arXiv 2412.06224 ↗First posted 9 Dec 2024cs.RO cs.CVcited by 146

On the map: Vision-language-action models → Generalist VLA models and benchmarks

Abstract (arXiv)

A practical navigation agent must be capable of handling a wide range of interaction demands, such as following instructions, searching objects, answering questions, tracking people, and more. Existing models for embodied navigation fall short of serving as practical generalists in the real world, as they are often constrained by specific task configurations or pre-defined maps with discretized waypoints. In this work, we present Uni-NaVid, the first video-based vision-language-action (VLA) model designed to unify diverse embodied navigation tasks and enable seamless navigation for mixed long-horizon tasks in unseen real-world environments. Uni-NaVid achieves this by harmonizing the input and output data configurations for all commonly used embodied navigation tasks and thereby integrating all tasks in one model. For training Uni-NaVid, we collect 3.6 million navigation data samples in total from four essential navigation sub-tasks and foster synergy in learning across them. Extensive experiments on comprehensive navigation benchmarks clearly demonstrate the advantages of unification modeling in Uni-NaVid and show it achieves state-of-the-art performance. Additionally, real-world experiments confirm the model's effectiveness and efficiency, shedding light on its strong generalizability.

Key facts. Cited by 146 papers in the map (data through 7 Oct 2026). Placed in Generalist VLA models and benchmarks, one of 13 directions in Vision-language-action models. Builds on 33 of its 125 references that are in the map. First posted on arXiv 9 Dec 2024.

Builds on

33 of 125 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul

Other references · 32

  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  2. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  3. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  4. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  5. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  6. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  7. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
  8. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  9. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
  10. 2018On Evaluation of Embodied Navigation Agentscited by 526
  11. 2022Visual Language Maps for Robot Navigationcited by 362
  12. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
20 more
  1. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
  2. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
  3. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  4. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  5. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  6. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
  7. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 135
  8. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 108
  9. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 87
  10. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  11. 2024HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigationcited by 71
  12. 2023OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNavcited by 68
  13. 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Modelscited by 66
  14. 2023A2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Modelscited by 61
  15. 2022CLIP-Nav: Using CLIP for Zero-Shot Vision-and-Language Navigationcited by 55
  16. 2024PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigatorscited by 55
  17. 2023Principles and Guidelines for Evaluating Social Robot Navigation Algorithmscited by 52
  18. 2024Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigationcited by 34
  19. 2024OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Roboticscited by 33
  20. 2021Deep Learning for Embodied Vision Navigation: A Surveycited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 48; 2026: 98.

146 papers.

Sort:

By area:

  1. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
  2. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
  3. 2026VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Modelcited by 104
  4. 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 80
  5. 2025VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuningcited by 61
  6. 2025Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigationcited by 50
  7. 2025ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledgecited by 49
  8. 2025SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulationcited by 46
  9. 2025NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidancecited by 46
  10. 2025TrackVLA: Embodied Visual Tracking in the Wildcited by 46
  11. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  12. 2025Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Taskscited by 43
38 more
  1. 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodimentscited by 42
  2. 2025Vision Language Action Models in Robotic Manipulation: A Systematic Reviewcited by 38
  3. 2025ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Contextcited by 34
  4. 2025OctoNav: Towards Generalist Embodied Navigationcited by 33
  5. 2025Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigationcited by 30
  6. 2025VLA-AN: An Efficient and Onboard Vision-Language-Action Framework for Aerial Navigation in Complex Environmentscited by 22
  7. 2025Dynam3D: Dynamic Layered 3D Tokens Empower VLM for Vision-and-Language Navigationcited by 21
  8. 2025VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluationcited by 17
  9. 2025AstraNav-World: World Model for Foresight Control and Consistencycited by 17
  10. 2025Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigationcited by 16
  11. 2025TrackVLA++: Unleashing Reasoning and Memory Capabilities in VLA Models for Embodied Visual Trackingcited by 16
  12. 2025DreamNav: A Trajectory-Based Imaginative Framework for Zero-Shot Vision-and-Language Navigationcited by 15
  13. 2025MobileVLA-R1: Reinforcing Vision-Language-Action for Mobile Robotscited by 15
  14. 2025NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Predictioncited by 15
  15. 2025NaviTrace: Evaluating Embodied Navigation of Vision-Language Modelscited by 13
  16. 2025MSGNav: Unleashing the Power of Multi-modal 3D Scene Graph for Zero-Shot Embodied Navigationcited by 13
  17. 2025ActiveVLN: Towards Active Exploration via Multi-Turn RL in Vision-and-Language Navigationcited by 11
  18. 2026VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memorycited by 10
  19. 2026Sparse Video Generation Propels Real-World Beyond-the-View Vision-Language Navigationcited by 10
  20. 2026Qwen-RobotNav Technical Report: A Scalable Navigation Model Designed for an Agentic Navigation Systemcited by 10
  21. 2026DeltaVLA: Prior-Guided Vision-Language-Action Models via World Knowledge Variationcited by 9
  22. 2025Nav-R2 Dual-Relation Reasoning for Generalizable Open-Vocabulary Object-Goal Navigationcited by 8
  23. 2026AgentVLN: Towards Agentic Vision-and-Language Navigationcited by 8
  24. 2025From reactive to cognitive: brain-inspired spatial intelligence for embodied agentscited by 7
  25. 2025Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoningcited by 7
  26. 2026Dual-Anchoring: Addressing State Drift in Vision-Language Navigationcited by 7
  27. 2026NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigationcited by 7
  28. 2025View Invariant Learning for Vision-Language Navigation in Continuous Environmentscited by 6
  29. 2025Imaginative World Modeling with Scene Graphs for Embodied Agent Navigationcited by 6
  30. 2026AwareVLN: Reasoning with Self-awareness for Vision-Language Navigationcited by 6
  31. 2026Bridging the 2D-3D Gap: A Hierarchical Semantic-Geometric Map for Vision Language Navigationcited by 6
  32. 2025D3D-VLP: Dynamic 3D Vision-Language-Planning Model for Embodied Grounding and Navigationcited by 5
  33. 2025VLN-MME: Diagnosing MLLMs as Language-guided Visual Navigation agentscited by 5
  34. 2026AirNav: A Large-Scale UAV Vision-and-Language Navigation Dataset with Natural and Diverse Instructionscited by 5
  35. 2026textscNaVIDA: Vision-Language Navigation with Inverse Dynamics Augmentationcited by 5
  36. 2026LongNav-R1: Horizon-Adaptive Multi-Turn RL for Long-Horizon VLA Navigationcited by 5
  37. 2026One Agent to Guide Them All: Empowering MLLMs for Vision-and-Language Navigation via Explicit World Representationcited by 5
  38. 2026CoMaTrack: Competitive Multi-Agent Game-Theoretic Tracking with Vision-Language-Action Modelscited by 5

50 most cited of 146 All 146 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation
  2. 2025Embodied Navigation Foundation Model
  3. 2025FiLM-Nav: Efficient and Generalizable Navigation via VLM Fine-tuning
  4. 2025NavBench: Probing Multimodal Large Language Models for Embodied Navigation
  5. 2025NavRAG: Generating User Demand Instructions for Embodied Navigation through Retrieval-Augmented LLM
  6. 2026ABot-N0: Technical Report on the VLA Foundation Model for Versatile Embodied Navigation
  7. 2026UniNav: A Unified World-Action Diffusion Model for Visual Navigation
  8. 2026NavDreamer: Video Models as Zero-Shot 3D Navigators