mlsanityData to 7 Oct 2026

MiniVLN: Efficient Vision-and-Language Navigation by Progressive Knowledge Distillation

arXiv 2409.18800 ↗First posted 27 Sep 2024cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

In recent years, Embodied Artificial Intelligence (Embodied AI) has advanced rapidly, yet the increasing size of models conflicts with the limited computational capabilities of Embodied AI platforms. To address this challenge, we aim to achieve both high model performance and practical deployability. Specifically, we focus on Vision-and-Language Navigation (VLN), a core task in Embodied AI. This paper introduces a two-stage knowledge distillation framework, producing a student model, MiniVLN, and showcasing the significant potential of distillation techniques in developing lightweight models. The proposed method aims to capture fine-grained knowledge during the pretraining phase and navigation-specific knowledge during the fine-tuning phase. Our findings indicate that the two-stage distillation approach is more effective in narrowing the performance gap between the teacher model and the student model compared to single-stage distillation. On the public R2R and REVERIE benchmarks, MiniVLN achieves performance on par with the teacher model while having only about 12% of the teacher model's parameter count.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 38 of its 41 references that are in the map. First posted on arXiv 27 Sep 2024.

Builds on

38 of 41 references.

References · 38

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  4. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  5. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  6. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  7. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  8. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  9. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  10. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  11. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  12. 2019Structured Knowledge Distillation for Dense Predictioncited by 277
26 more
  1. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
  2. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  3. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 129
  4. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 118
  5. 2022Cross-Image Relational Knowledge Distillation for Semantic Segmentationcited by 117
  6. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  7. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
  8. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  9. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 71
  10. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  11. 2021Less is More: Generating Grounded Navigation Instructions from Landmarkscited by 62
  12. 2022Target-Driven Structured Transformer Planner for Vision-Language Navigationcited by 58
  13. 2021Online Knowledge Distillation for Efficient Pose Estimationcited by 55
  14. 2024Vision-and-Language Navigation via Causal Learningcited by 51
  15. 2022Counterfactual Cycle-Consistent Learning for Instruction Following and Generation in Vision-Language Navigationcited by 50
  16. 2023CrossKD: Cross-Head Knowledge Distillation for Object Detectioncited by 50
  17. 2022Learning from Unlabeled 3D Environments for Vision-and-Language Navigationcited by 49
  18. 2022ADAPT: Vision-Language Navigation with Modality-Aligned Action Promptscited by 45
  19. 2022PKD: General Distillation Framework for Object Detectors via Pearson Correlation Coefficientcited by 41
  20. 2023Learning Navigational Visual Representations with Semantic Map Supervisioncited by 40
  21. 2023KERM: Knowledge Enhanced Reasoning for Vision-and-Language Navigationcited by 38
  22. 2023GeoVLN: Learning Geometry-Enhanced Visual Representation with Slot Attention for Vision-and-Language Navigationcited by 24
  23. 2023VLN-PETL: Parameter-Efficient Transfer Learning for Vision-and-Language Navigationcited by 22
  24. 2023A Dual Semantic-Aware Recurrent Global-Adaptive Network For Vision-and-Language Navigationcited by 18
  25. 2023Kefa: A Knowledge Enhanced and Fine-grained Aligned Speaker for Navigation Instruction Generationcited by 11
  26. 2024MAGIC: Meta-Ability Guided Interactive Chain-of-Distillation for Effective-and-Efficient Vision-and-Language Navigationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025UrbanNav: Learning Language-Guided Urban Navigation from Web-Scale Human Trajectoriescited by 6
  2. 2025Weakly-supervised VLM-guided Partial Contrastive Learning for Visual Language Navigationcited by 4
  3. 2025Think Hierarchically, Act Dynamically: Hierarchical Multi-modal Fusion and Reasoning for Vision-and-Language Navigationcited by 3
  4. 2025Walk and Read Less: Improving the Efficiency of Vision-and-Language Navigation via Tuning-Free Multimodal Token Pruningcited by 0
  5. 2026PM-Nav: Priori-Map Guided Embodied Navigation in Functional Buildingscited by 0
  6. 2026Query, Align, and Distill: Navigation-Aware Cross-Modal Interaction for Efficient Vision-and-Language Navigationcited by 0