mlsanityData to 7 Oct 2026

Curriculum Learning for Vision-and-Language Navigation

arXiv 2111.07228 ↗First posted 14 Nov 2021cs.LG cs.AI cs.CLcited by 18

Not on the map. Why?

Abstract (arXiv)

Vision-and-Language Navigation (VLN) is a task where an agent navigates in an embodied indoor environment under human instructions. Previous works ignore the distribution of sample difficulty and we argue that this potentially degrade their agent performance. To tackle this issue, we propose a novel curriculum-based training paradigm for VLN tasks that can balance human prior knowledge and agent learning progress about training samples. We develop the principle of curriculum design and re-arrange the benchmark Room-to-Room (R2R) dataset to make it suitable for curriculum training. Experiments show that our method is model-agnostic and can significantly improve the performance, the generalizability, and the training efficiency of current state-of-the-art navigation agents without increasing model complexity.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 29 of its 39 references that are in the map. First posted on arXiv 14 Nov 2021.

Builds on

29 of 39 references.

References · 29

  1. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  2. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  3. 2018On Evaluation of Embodied Navigation Agentscited by 526
  4. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  5. 2018How Does Batch Normalization Help Optimization?cited by 355
  6. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  7. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  8. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  9. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 285
  10. 2017Automated Curriculum Learning for Neural Networkscited by 251
  11. 2019Vision-and-Dialog Navigationcited by 246
  12. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
17 more
  1. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  2. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
  3. 2019Competence-based Curriculum Learning for Neural Machine Translationcited by 201
  4. 2017Teacher-Student Curriculum Learningcited by 197
  5. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 177
  6. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  7. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 173
  8. 2019Stay on the Path: Instruction Fidelity in Vision-and-Language Navigationcited by 129
  9. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  10. 2018Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigationcited by 84
  11. 2021WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Trainingcited by 84
  12. 2019Robust Navigation with Language Pretraining and Stochastic Samplingcited by 77
  13. 2020Sub-Instruction Aware Vision-and-Language Navigationcited by 64
  14. 2020BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Stepscited by 49
  15. 2020Environment-agnostic Multitask Learning for Natural Language Grounded Navigationcited by 43
  16. 2019Multi-modal Discriminative Model for Vision-and-Language Navigationcited by 17
  17. 2019Measuring the Data Efficiency of Deep Learning Methodscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 1; 2024: 5; 2025: 3; 2026: 4.

18 papers.

Sort:

By area:

  1. 2022Multimodal Learning with Transformers: A Surveycited by 399
  2. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  3. 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Modelscited by 100
  4. 2022Target-Driven Structured Transformer Planner for Vision-Language Navigationcited by 58
  5. 2024Navigating Complexity: Toward Lossless Graph Condensation via Expanding Window Matchingcited by 28
  6. 2025Object Detection with Multimodal Large Vision-Language Models: An In-depth Reviewcited by 14
  7. 2023Graph based Environment Representation for Vision-and-Language Navigation in Continuous Environmentscited by 9
  8. 2024Vision-Language Navigation with Embodied Intelligence: A Surveycited by 9
  9. 2026SpatialNav: Leveraging Spatial Scene Graphs for Zero-Shot Vision-and-Language Navigationcited by 8
  10. 2024DELAN: Dual-Level Alignment for Vision-and-Language Navigation by Cross-Modal Contrastive Learningcited by 7
  11. 2025NavQ: Learning a Q-Model for Foresighted Vision-and-Language Navigationcited by 6
  12. 2026Trajectory-Diversity-Driven Robust Vision-and-Language Navigationcited by 4
6 more
  1. 2024Can Vision Language Models Learn from Visual Demonstrations of Ambiguous Spatial Reasoning?cited by 3
  2. 2022InDistill: Information flow-preserving knowledge distillation for model compressioncited by 2
  3. 2022Vision-and-Language Pretrainingcited by 2
  4. 2026SpatialAnt: Autonomous Zero-Shot Robot Navigation via Active Scene Reconstruction and Visual Anticipationcited by 2
  5. 2025Can an Easy-to-Hard Curriculum Make Reasoning Emerge in Small Language Models? Evidence from a Four-Stage Curriculum on GPT-2cited by 1
  6. 2026From Region Arrival to Instance-Level Grounding in Vision-and-Language Navigationcited by 0