mlsanityData to 7 Oct 2026

Behavioral Analysis of Vision-and-Language Navigation Agents

arXiv 2307.10790 ↗First posted 20 Jul 2023cs.CV cs.ROcited by 8

Not on the map. Why?

Abstract (arXiv)

To be successful, Vision-and-Language Navigation (VLN) agents must be able to ground instructions to actions based on their surroundings. In this work, we develop a methodology to study agent behavior on a skill-specific basis -- examining how well existing agents ground instructions about stopping, turning, and moving towards specified objects or rooms. Our approach is based on generating skill-specific interventions and measuring changes in agent predictions. We present a detailed case study analyzing the behavior of a recent agent and then compare multiple agents in terms of skill-specific competency scores. This analysis suggests that biases from training have lasting effects on agent behavior and that existing models are able to ground simple referring expressions. Our comparisons between models show that skill-specific scores correlate with improvements in overall VLN task performance.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 19 of its 24 references that are in the map. First posted on arXiv 20 Jul 2023.

Builds on

19 of 24 references.

References · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  3. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  4. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  5. 2018On Evaluation of Embodied Navigation Agentscited by 526
  6. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  7. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  8. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  9. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  10. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  11. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  12. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
7 more
  1. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 129
  2. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  3. 2019General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warpingcited by 102
  4. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  5. 2022A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learningcited by 60
  6. 2021SOAT: A Scene- and Object-Aware Transformer for Vision-and-Language Navigationcited by 53
  7. 2021Diagnosing Vision-and-Language Navigation: What Really Matterscited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 1; 2026: 1.

8 papers.

  1. 2023Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigationcited by 25
  2. 2024Instance-aware Exploration-Verification-Exploitation for Instance ImageGoal Navigationcited by 15
  3. 2025Do Visual Imaginations Improve Vision-and-Language Navigation Agents?cited by 14
  4. 2024Mind the Error! Detection and Localization of Instruction Errors in Vision-and-Language Navigationcited by 11
  5. 2024DELAN: Dual-Level Alignment for Vision-and-Language Navigation by Cross-Modal Contrastive Learningcited by 7
  6. 2024Tell Me Where You Are: Multimodal LLMs Meet Place Recognitioncited by 7
  7. 2024Hijacking Vision-and-Language Navigation Agents with Adversarial Environmental Attackscited by 5
  8. 2026IntentionNav: A Benchmark for Intent-Driven Object Navigation from Implicit Human Instructioncited by 1