# Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environments

arXiv: https://arxiv.org/abs/2109.15207 · HTML: https://mlsanity.com/paper/2109.15207/

## Key facts

- Cited by 94 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 28 references that are in the map.
- First posted on arXiv 30 Sep 2021.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2004.02857/) (2020)
- [Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding](https://mlsanity.com/paper/2010.07954/) (2020)
- [DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames](https://mlsanity.com/paper/1911.00357/) (2019)
- [Speaker-Follower Models for Vision-and-Language Navigation](https://mlsanity.com/paper/1806.02724/) (2018)
- [Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation](https://mlsanity.com/paper/1811.10092/) (2018)
- [Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout](https://mlsanity.com/paper/1904.04195/) (2019)
- [Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments](https://mlsanity.com/paper/1811.12354/) (2018)
- [Self-Monitoring Navigation Agent via Auxiliary Progress Estimation](https://mlsanity.com/paper/1901.03035/) (2019)
- [Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation](https://mlsanity.com/paper/1905.12255/) (2019)
- [Topological Planning with Transformers for Vision-and-Language Navigation](https://mlsanity.com/paper/2012.05292/) (2020)
- [General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warping](https://mlsanity.com/paper/1907.05446/) (2019)
- [Object-and-Action Aware Model for Visual Language Navigation](https://mlsanity.com/paper/2007.14626/) (2020)
- [Sub-Instruction Aware Vision-and-Language Navigation](https://mlsanity.com/paper/2004.02707/) (2020)
- [Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation](https://mlsanity.com/paper/1906.00347/) (2019)
- [BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps](https://mlsanity.com/paper/2005.04625/) (2020)
- [Chasing Ghosts: Instruction Following as Bayesian State Tracking](https://mlsanity.com/paper/1907.02022/) (2019)
- [Retouchdown: Adding Touchdown to StreetLearn as a Shareable Resource for Language Grounding Tasks in Street View](https://mlsanity.com/paper/2001.03671/) (2020)

## Built on by (most cited first, 30 of 94)

- [NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation](https://mlsanity.com/paper/2402.15852/) (2024)
- [NaVILA: Legged Robot Vision-Language-Action Model for Navigation](https://mlsanity.com/paper/2412.04453/) (2024)
- [ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2304.03047/) (2023)
- [Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Tasks](https://mlsanity.com/paper/2412.06224/) (2024)
- [Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigation](https://mlsanity.com/paper/2203.02764/) (2022)
- [GridMM: Grid Memory Map for Vision-and-Language Navigation](https://mlsanity.com/paper/2307.12907/) (2023)
- [AerialVLN: Vision-and-Language Navigation for UAVs](https://mlsanity.com/paper/2308.06735/) (2023)
- [StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modeling](https://mlsanity.com/paper/2507.05240/) (2025)
- [Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions](https://mlsanity.com/paper/2203.12667/) (2022)
- [Scaling Data Generation in Vision-and-Language Navigation](https://mlsanity.com/paper/2307.15644/) (2023)
- [Cross-modal Map Learning for Vision and Language Navigation](https://mlsanity.com/paper/2203.05137/) (2022)
- [JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigation](https://mlsanity.com/paper/2509.22548/) (2025)
- [Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2204.09667/) (2022)
- [Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigation](https://mlsanity.com/paper/2210.07506/) (2022)
- [DREAMWALKER: Mental Planning for Continuous Vision-Language Navigation](https://mlsanity.com/paper/2308.07498/) (2023)
- [Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Models](https://mlsanity.com/paper/2407.07035/) (2024)
- [Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigation](https://mlsanity.com/paper/2407.05890/) (2024)
- [VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning](https://mlsanity.com/paper/2506.17221/) (2025)
- [A2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Models](https://mlsanity.com/paper/2308.07997/) (2023)
- [Learning from Unlabeled 3D Environments for Vision-and-Language Navigation](https://mlsanity.com/paper/2208.11781/) (2022)
- [Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2412.10137/) (2024)
- [Retrospectives on the Embodied AI Workshop](https://mlsanity.com/paper/2210.06849/) (2022)
- [Improving Vision-and-Language Navigation by Generating Future-View Image Semantics](https://mlsanity.com/paper/2304.04907/) (2023)
- [Openfly: A comprehensive platform for aerial vision-language navigation](https://mlsanity.com/paper/2502.18041/) (2025)
- [NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2506.23468/) (2025)
- [Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks](https://mlsanity.com/paper/2503.21696/) (2025)
- [Learning Navigational Visual Representations with Semantic Map Supervision](https://mlsanity.com/paper/2307.12335/) (2023)
- [CityNavAgent: Aerial Vision-and-Language Navigation with Hierarchical Semantic Planning and Global Memory](https://mlsanity.com/paper/2505.05622/) (2025)
- [CorrectNav: Self-Correction Flywheel Empowers Vision-Language-Action Navigation Model](https://mlsanity.com/paper/2508.10416/) (2025)
- [Last-Mile Embodied Visual Navigation](https://mlsanity.com/paper/2211.11746/) (2022)

## Abstract (arXiv)

In the Vision-and-Language Navigation (VLN) task an embodied agent navigates a 3D environment, following natural language instructions. A challenge in this task is how to handle 'off the path' scenarios where an agent veers from a reference path. Prior work supervises the agent with actions based on the shortest path from the agent's location to the goal, but such goal-oriented supervision is often not in alignment with the instruction. Furthermore, the evaluation metrics employed by prior work do not measure how much of a language instruction the agent is able to follow. In this work, we propose a simple and effective language-aligned supervision scheme, and a new metric that measures the number of sub-instructions the agent has completed during navigation.