mlsanityData to 14 Oct 2026

LeLaN: Learning A Language-Conditioned Navigation Policy from In-the-Wild Videos

arXiv 2410.03603 ↗First posted 4 Oct 2024cs.ROcited by 32

Not on the map. Why?

Abstract (arXiv)

The world is filled with a wide variety of objects. For robots to be useful, they need the ability to find arbitrary objects described by people. In this paper, we present LeLaN(Learning Language-conditioned Navigation policy), a novel approach that consumes unlabeled, action-free egocentric data to learn scalable, language-conditioned object navigation. Our framework, LeLaN leverages the semantic knowledge of large vision-language models, as well as robotic foundation models, to label in-the-wild data from a variety of indoor and outdoor environments. We label over 130 hours of data collected in real-world indoor and outdoor environments, including robot observations, YouTube video tours, and human walking data. Extensive experiments with over 1000 real-world trials show that our approach enables training a policy from unlabeled action-free videos that outperforms state-of-the-art robot navigation methods, while being capable of inference at 4 times their speed on edge compute. We open-source our models, datasets and provide supplementary videos on our project page (https://learning-language-navigation.github.io/).

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). First posted on arXiv 4 Oct 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 13; 2026: 14.

32 papers.

Sort:

By area:

  1. 2024Steering Your Generalists: Improving Robotic Foundation Models via Value Guidancecited by 106
  2. 2024CityWalker: Learning Embodied Urban Navigation from Web-Scale Videoscited by 41
  3. 2025OmniVLA: An Omni-Modal Vision-Language-Action Model for Robot Navigationcited by 38
  4. 2025CAST: Counterfactual Labels Improve Instruction Following in Vision-Language-Action Modelscited by 29
  5. 2024GHIL-Glue: Hierarchical Control with Filtered Subgoal Imagescited by 15
  6. 2024Scaling Robot Policy Learning via Zero-Shot Labeling with Foundation Modelscited by 14
  7. 2025Learning to Drive Anywhere with Model-Based Reannotationcited by 13
  8. 2024FlowNav: Combining Flow Matching and Depth Priors for Efficient Navigationcited by 12
  9. 2025Mobi-π: Mobilizing Your Robot Learning Policycited by 10
  10. 2025SocialNav-SUB: Benchmarking VLMs for Scene Understanding in Social Robot Navigationcited by 10
  11. 2026AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edgecited by 10
  12. 2025CREStE: Scalable Mapless Navigation with Internet Scale Priors and Counterfactual Guidancecited by 9
20 more
  1. 2026NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigationcited by 8
  2. 2025UrbanNav: Learning Language-Guided Urban Navigation from Web-Scale Human Trajectoriescited by 6
  3. 2025MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learningcited by 3
  4. 2026Bridging the Indoor-Outdoor Gap: Vision-Centric Instruction-Guided Embodied Navigation for the Last Meterscited by 3
  5. 2026WildOS: Open-Vocabulary Object Search in the Wildcited by 3
  6. 2026From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Datacited by 3
  7. 2025VENTURA: Adapting Image Diffusion Models for Unified Task Conditioned Navigationcited by 2
  8. 2025Efficient Navigation in Unknown Indoor Environments with Vision-Language Modelscited by 2
  9. 2025Target-Bench: Can Video World Models Achieve Mapless Path Planning with Semantic Targets?cited by 2
  10. 2025DIV-Nav: Open-Vocabulary Spatial Relationships for Multi-Object Navigationcited by 1
  11. 2026TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Modelcited by 1
  12. 2025Stereo4DWalker: Learning 4D-aware Embodied Urban Navigation from Internet Stereo Videoscited by 0
  13. 2026Foresight: Iterative Reasoning About Clues that Matter for Navigationcited by 0
  14. 2026Vision-Language Models for Deployable Social Robot Navigation: Bridging Semantic Reasoning and Low-Level Controlcited by 0
  15. 2026GemNav: Discrete-Token Visual Robot Navigation using a Multimodal Large Language Modelcited by 0
  16. 2026CrossTracer: Cross-Embodiment Navigation via VLA Model Reasoning and Trace Residuals Adaptingcited by 0
  17. 2026Goal-oriented Navigation Instruction Generation with Tour Video Priorscited by 0
  18. 2026RECAST: Recasting Vision-Language Semantics into an Actionable Cost Map for Robot Navigationcited by 0
  19. 2026Learning Social Navigation from Internet Videos in the Policy State Spacecited by 0
  20. 2026Training Generalist Navigation Policies via Action Self-Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation
  2. 2026VEGA: Learning Navigation VLAs from In-the-Wild Egocentric Video with Geometric Trajectory Supervision
  3. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Tasks
  4. 2025VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning
  5. 2025FiLM-Nav: Efficient and Generalizable Navigation via VLM Fine-tuning
  6. 2026ImagiNav: Scalable Embodied Navigation via Generative Visual Prediction and Inverse Dynamics
  7. 2026Language-Conditioned World Modeling for Visual Navigation
  8. 2026CoFL-S: Spatially Queryable Sector Flow Fields for Local Language-Conditioned Navigation