# LeLaN: Learning A Language-Conditioned Navigation Policy from In-the-Wild Videos

arXiv: https://arxiv.org/abs/2410.03603 · HTML: https://mlsanity.com/paper/2410.03603/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Oct 2024.

## Built on by (most cited first, 30 of 32)

- [Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance](https://mlsanity.com/paper/2410.13816/) (2024)
- [CityWalker: Learning Embodied Urban Navigation from Web-Scale Videos](https://mlsanity.com/paper/2411.17820/) (2024)
- [OmniVLA: An Omni-Modal Vision-Language-Action Model for Robot Navigation](https://mlsanity.com/paper/2509.19480/) (2025)
- [CAST: Counterfactual Labels Improve Instruction Following in Vision-Language-Action Models](https://mlsanity.com/paper/2508.13446/) (2025)
- [GHIL-Glue: Hierarchical Control with Filtered Subgoal Images](https://mlsanity.com/paper/2410.20018/) (2024)
- [Scaling Robot Policy Learning via Zero-Shot Labeling with Foundation Models](https://mlsanity.com/paper/2410.17772/) (2024)
- [Learning to Drive Anywhere with Model-Based Reannotation](https://mlsanity.com/paper/2505.05592/) (2025)
- [FlowNav: Combining Flow Matching and Depth Priors for Efficient Navigation](https://mlsanity.com/paper/2411.09524/) (2024)
- [Mobi-π: Mobilizing Your Robot Learning Policy](https://mlsanity.com/paper/2505.23692/) (2025)
- [SocialNav-SUB: Benchmarking VLMs for Scene Understanding in Social Robot Navigation](https://mlsanity.com/paper/2509.08757/) (2025)
- [AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edge](https://mlsanity.com/paper/2602.13476/) (2026)
- [CREStE: Scalable Mapless Navigation with Internet Scale Priors and Counterfactual Guidance](https://mlsanity.com/paper/2503.03921/) (2025)
- [NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation](https://mlsanity.com/paper/2606.13494/) (2026)
- [UrbanNav: Learning Language-Guided Urban Navigation from Web-Scale Human Trajectories](https://mlsanity.com/paper/2512.09607/) (2025)
- [MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learning](https://mlsanity.com/paper/2510.03142/) (2025)
- [Bridging the Indoor-Outdoor Gap: Vision-Centric Instruction-Guided Embodied Navigation for the Last Meters](https://mlsanity.com/paper/2602.06427/) (2026)
- [WildOS: Open-Vocabulary Object Search in the Wild](https://mlsanity.com/paper/2602.19308/) (2026)
- [From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data](https://mlsanity.com/paper/2606.00054/) (2026)
- [VENTURA: Adapting Image Diffusion Models for Unified Task Conditioned Navigation](https://mlsanity.com/paper/2510.01388/) (2025)
- [Efficient Navigation in Unknown Indoor Environments with Vision-Language Models](https://mlsanity.com/paper/2510.04991/) (2025)
- [Target-Bench: Can Video World Models Achieve Mapless Path Planning with Semantic Targets?](https://mlsanity.com/paper/2511.17792/) (2025)
- [DIV-Nav: Open-Vocabulary Spatial Relationships for Multi-Object Navigation](https://mlsanity.com/paper/2510.16518/) (2025)
- [TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model](https://mlsanity.com/paper/2609.09158/) (2026)
- [Stereo4DWalker: Learning 4D-aware Embodied Urban Navigation from Internet Stereo Videos](https://mlsanity.com/paper/2512.10956/) (2025)
- [Foresight: Iterative Reasoning About Clues that Matter for Navigation](https://mlsanity.com/paper/2606.12550/) (2026)
- [Vision-Language Models for Deployable Social Robot Navigation: Bridging Semantic Reasoning and Low-Level Control](https://mlsanity.com/paper/2606.28760/) (2026)
- [GemNav: Discrete-Token Visual Robot Navigation using a Multimodal Large Language Model](https://mlsanity.com/paper/2607.06882/) (2026)
- [CrossTracer: Cross-Embodiment Navigation via VLA Model Reasoning and Trace Residuals Adapting](https://mlsanity.com/paper/2608.06688/) (2026)
- [Goal-oriented Navigation Instruction Generation with Tour Video Priors](https://mlsanity.com/paper/2608.08596/) (2026)
- [RECAST: Recasting Vision-Language Semantics into an Actionable Cost Map for Robot Navigation](https://mlsanity.com/paper/2609.32595/) (2026)

## Abstract (arXiv)

The world is filled with a wide variety of objects. For robots to be useful, they need the ability to find arbitrary objects described by people. In this paper, we present LeLaN(Learning Language-conditioned Navigation policy), a novel approach that consumes unlabeled, action-free egocentric data to learn scalable, language-conditioned object navigation. Our framework, LeLaN leverages the semantic knowledge of large vision-language models, as well as robotic foundation models, to label in-the-wild data from a variety of indoor and outdoor environments. We label over 130 hours of data collected in real-world indoor and outdoor environments, including robot observations, YouTube video tours, and human walking data. Extensive experiments with over 1000 real-world trials show that our approach enables training a policy from unlabeled action-free videos that outperforms state-of-the-art robot navigation methods, while being capable of inference at 4 times their speed on edge compute. We open-source our models, datasets and provide supplementary videos on our project page (https://learning-language-navigation.github.io/).