mlsanityData to 14 Oct 2026

NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidance

arXiv 2505.08712 ↗First posted 13 May 2025cs.ROcited by 49

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Learning to navigate in dynamic and complex open-world environments is a critical yet challenging capability for autonomous robots. Existing approaches often rely on cascaded modular frameworks, which require extensive hyperparameter tuning or learning from limited real-world demonstration data. In this paper, we propose Navigation Diffusion Policy (NavDP), an end-to-end network trained solely in simulation that enables zero-shot sim-to-real transfer across diverse environments and robot embodiments. The core of NavDP is a unified transformer-based architecture that jointly learns trajectory generation and trajectory evaluation, both conditioned solely on local RGB-D observation. By learning to predict critic values for contrastive trajectory samples, our proposed approach effectively leverages supervision from privileged information available in simulation, thereby fostering accurate spatial understanding and enabling the distinction between safe and dangerous behaviors. To support this, we develop an efficient data generation pipeline in simulation and construct a large-scale dataset encompassing over one million meters of navigation experience across 3,000 scenes. Empirical experiments in both simulated and real-world environments demonstrate that NavDP significantly outperforms prior state-of-the-art methods. Furthermore, we identify key factors influencing the generalization performance of NavDP. The dataset and code are publicly available at https://wzcai99.github.io/navigation-diffusion-policy.github.io.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 51 of its 57 references that are in the map. First posted on arXiv 13 May 2025.

Builds on

51 of 57 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillation28 Oct
  2. 2024Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learning1 Jul
  3. 2024Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillation13 May
  4. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representations16 Feb
  5. 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration11 Oct
  6. 2023Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learning4 Jul
  7. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar

Other references · 44

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  5. 2024Depth Anything V2cited by 1,422
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  7. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  8. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  9. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
  10. 2023Scaffold-GS: Structured 3D Gaussians for View-Adaptive Renderingcited by 695
  11. 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 616
  12. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
32 more
  1. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
  2. 2023Gaussian Splatting SLAMcited by 415
  3. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 380
  4. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 362
  5. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
  6. 2023ViNT: A Foundation Model for Visual Navigationcited by 222
  7. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 211
  8. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 205
  9. 2022GNM: A General Navigation Model to Drive Any Robotcited by 171
  10. 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 167
  11. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 160
  12. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 156
  13. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 144
  14. 2022Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigationcited by 127
  15. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 121
  16. 2022Habitat-Matterport 3D Semantics Datasetcited by 110
  17. 2024Automated Creation of Digital Cousins for Robust Policy Learningcited by 91
  18. 2023Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skillcited by 77
  19. 2023XSkill: Cross Embodiment Skill Discoverycited by 75
  20. 2023SACSoN: Scalable Autonomous Control for Social Navigationcited by 61
  21. 2024GRUtopia: Dream General Robots in a City at Scalecited by 59
  22. 2024PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigatorscited by 58
  23. 2019Deep Visual MPC-Policy Learning for Navigationcited by 56
  24. 2020EGO-Planner: An ESDF-free Gradient-based Local Planner for Quadrotorscited by 55
  25. 2024DiffuseLoco: Real-Time Legged Locomotion Control with Diffusion from Offline Datasetscited by 49
  26. 2023SPOC: Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real Worldcited by 48
  27. 2023ViPlanner: Visual Semantic Imperative Learning for Local Navigationcited by 46
  28. 2021Planar Robot Casting with Real2Sim2Real Self-Supervised Learningcited by 39
  29. 2023iPlanner: Imperative Path Planningcited by 37
  30. 2024The One RING: a Robotic Indoor Navigation Generalistcited by 16
  31. 2020Sim2Real2Sim: Bridging the Gap Between Simulation and Real-World in Flexible Object Manipulationcited by 14
  32. 2024Aim My Robot: Precision Local Navigation to Any Objectcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 35.

49 papers.

Sort:

By area:

  1. 2025Rethinking the Embodied Gap in Vision-and-Language Navigation: A Holistic Study of Physical and Visual Disparitiescited by 25
  2. 2025InternScenes: A Large-scale Simulatable Indoor Scene Dataset with Realistic Layoutscited by 20
  3. 2025VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluationcited by 19
  4. 2025DreamNav: A Trajectory-Based Imaginative Framework for Zero-Shot Vision-and-Language Navigationcited by 17
  5. 2026TextOp: Real-time Interactive Text-Driven Humanoid Robot Motion Generation and Controlcited by 14
  6. 2026TIC-VLA: A Think-in-Control Vision-Language-Action Model for Robot Navigation in Dynamic Environmentscited by 10
  7. 2025LoGoPlanner: Localization Grounded Navigation Policy with Metric-aware Visual Geometrycited by 6
  8. 2026textscNaVIDA: Vision-Language Navigation with Inverse Dynamics Augmentationcited by 5
  9. 2025MetricNet: Recovering Metric Scale in Generative Navigation Policiescited by 3
  10. 2025MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learningcited by 3
  11. 2025UrbanVLA: A Vision-Language-Action Model for Urban Micromobilitycited by 3
  12. 2025MG-Nav: Dual-Scale Visual Navigation via Sparse Spatial Memorycited by 3
37 more
  1. 2026SynthVerse: A Large-Scale Diverse Synthetic Dataset for Point Trackingcited by 3
  2. 2026NavOL: Navigation Policy with Online Imitation Learningcited by 3
  3. 2026Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigationcited by 3
  4. 2026HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harnesscited by 3
  5. 2025DiffVL: Diffusion-Based Visual Localization on 2D Maps via BEV-Conditioned GPS Denoisingcited by 2
  6. 2025CostNav: A Navigation Benchmark for Real-World Economic-Cost Evaluation of Physical AI Agentscited by 2
  7. 2026A Review of Online Diffusion Policy RL Algorithms for Scalable Robotic Controlcited by 2
  8. 2026Self-Imitated Diffusion Policy for Efficient and Robust Visual Navigationcited by 2
  9. 2026FLUX: Accelerating Cross-Embodiment Generative Navigation Policies via Rectified Flow and Static-to-Dynamic Learningcited by 2
  10. 2026Beyond Imitation: Reinforcement Learning Fine-Tuning for Adaptive Diffusion Navigation Policiescited by 2
  11. 2026UniNav: A Unified World-Action Diffusion Model for Visual Navigationcited by 2
  12. 2025Sight Over Site: Perception-Aware Reinforcement Learning for Efficient Robotic Inspectioncited by 1
  13. 2025Deep Active Inference with Diffusion Policy and Multiple Timescale World Model for Real-World Exploration and Navigationcited by 1
  14. 2026What Limits Vision-and-Language Navigation ?cited by 1
  15. 2026X-NavDP: Generalizing Navigation Diffusion Policy to Novel Behavior and Embodiments with Group Q-score Reweighted Matchingcited by 1
  16. 2026Steering Generative Robot Policies with Lexicographic Preferencescited by 1
  17. 2025ISS Policy : Scalable Diffusion Policy with Implicit Scene Supervisioncited by 0
  18. 2026LabBuilder: Protocol-Grounded 3D Layout Generation for Interactable and Safe Laboratorycited by 0
  19. 2026AgenticDiffusion: Multi-View Reasoning with View-Conditioned Diffusion Planning for Vision-Based UAV Navigationcited by 0
  20. 2026From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigationcited by 0
  21. 2026VISTA: Scale-Aware Visual Navigation via Action History Conditioningcited by 0
  22. 2026EffiNav: Fusing Depth and Vision-Language for Efficient Object Goal Navigationcited by 0
  23. 2026RoamFlow: Reinforcement-Aligned One-Step Action MeanFlow Policy for Image-Goal Navigationcited by 0
  24. 2026Learning to Navigate with Minimal Parameters: Decomposing Visual Navigation Through Closed-Form Geometric Interfacescited by 0
  25. 2026NavCMPO: Critic-Guided MeanFlow Policy Optimization for Adaptive Navigationcited by 0
  26. 2026Predictive Training with Latent Imagination for Visual Quadruped Navigationcited by 0
  27. 2026Learning Adaptive Safety Margins for Visual Navigationcited by 0
  28. 2026EA-Nav: Learning Safe Visual Navigation Policies with Embodiment Awarenesscited by 0
  29. 2026CrossTracer: Cross-Embodiment Navigation via VLA Model Reasoning and Trace Residuals Adaptingcited by 0
  30. 2026MulDP: Multimodal Diffusion Policy for Autonomous Quadruped Parkour Navigation across Complex Terrainscited by 0
  31. 2026WAVE-Go: World-Model Navigation with Adaptive Execution for Wheel-Legged Robotscited by 0
  32. 2026Learning Reliable Parking Policies via Offline Reinforcement Learning with Quantized Action Representationscited by 0
  33. 2026Skytopia: Monocular Drone Navigation with Action-Conditioned Latent World Modelscited by 0
  34. 2026RECAST: Recasting Vision-Language Semantics into an Actionable Cost Map for Robot Navigationcited by 0
  35. 2026World SLAM Model: Joint World Modeling for SLAM and Navigationcited by 0
  36. 2026InfraVLA: Extending Vision-Language-Action Navigation with Infrastructure Camerascited by 0
  37. 2026DR-APO: Decoupled-Representation Advantage-Guided Policy Optimization from Continuous Control to Embodied Navigationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025NaviDiffusor: Cost-Guided Diffusion Model for Visual Navigation
  2. 2026NavRL++: A System-Level Framework for Improving Sim-to-Real Transfer in Reinforcement Learning-Based Robot Navigation
  3. 2024NavFormer: A Transformer Architecture for Robot Target-Driven Navigation in Unknown and Dynamic Environments
  4. 2024LDP: A Local Diffusion Planner for Efficient Robot Navigation and Collision Avoidance
  5. 2026NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation
  6. 2025Exploration and Adaptation in Non-Stationary Tasks with Diffusion Policies
  7. 2024Navigation World Models
  8. 2026AR Forcing: Towards Long-Horizon Robot Navigation World Model