mlsanityData to 14 Oct 2026

Explore the Potential Performance of Vision-and-Language Navigation Model: a Snapshot Ensemble Method

arXiv 2111.14267 ↗First posted 28 Nov 2021cs.CV cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Vision-and-Language Navigation (VLN) is a challenging task in the field of artificial intelligence. Although massive progress has been made in this task over the past few years attributed to breakthroughs in deep vision and language models, it remains tough to build VLN models that can generalize as well as humans. In this paper, we provide a new perspective to improve VLN models. Based on our discovery that snapshots of the same VLN model behave significantly differently even when their success rates are relatively the same, we propose a snapshot-based ensemble solution that leverages predictions among multiple snapshots. Constructed on the snapshots of the existing state-of-the-art (SOTA) model circlearrowrightBERT and our past-action-aware modification, our proposed ensemble achieves the new SOTA performance in the R2R dataset challenge in Navigation Error (NE) and Success weighted by Path Length (SPL).

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 35 of its 43 references that are in the map. First posted on arXiv 28 Nov 2021.

Builds on

35 of 43 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  4. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  5. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  7. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  8. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,057
  9. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  10. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
  11. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  12. 2016Semantic Scene Completion from a Single Depth Imagecited by 472
23 more
  1. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  2. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  3. 2017Snapshot Ensembles: Train 1, get M for freecited by 350
  4. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
  5. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
  6. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 244
  7. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 231
  8. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 179
  9. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  10. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 176
  11. 2019Stay on the Path: Instruction Fidelity in Vision-and-Language Navigationcited by 130
  12. 2020Topological Planning with Transformers for Vision-and-Language Navigationcited by 116
  13. 2018Building Generalizable Agents with a Realistic and Rich 3D Environmentcited by 109
  14. 2019The Regretful Agent: Heuristic-Aided Navigation through Progress Estimationcited by 105
  15. 2020Language and Visual Entity Relationship Graph for Agent Navigationcited by 100
  16. 2021Structured Scene Memory for Vision-Language Navigationcited by 95
  17. 2019Robust Navigation with Language Pretraining and Stochastic Samplingcited by 77
  18. 2021Vision-Language Navigation with Random Environmental Mixupcited by 70
  19. 2020Sub-Instruction Aware Vision-and-Language Navigationcited by 66
  20. 2019Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigationcited by 55
  21. 2018CHALET: Cornell House Agent Learning Environmentcited by 38
  22. 2013Horizontal and Vertical Ensemble with Deep Representation for Classificationcited by 22
  23. 2017HoME: a Household Multimodal Environmentcited by 20

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 1.

4 papers.

  1. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
  2. 20221st Place Solutions for RxR-Habitat Vision-and-Language Navigation Competition (CVPR 2022)cited by 34
  3. 2023Grounded Entity-Landmark Adaptive Pre-training for Vision-and-Language Navigationcited by 27
  4. 2024Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheelcited by 20