mlsanityData to 14 Oct 2026

MAGIC: Meta-Ability Guided Interactive Chain-of-Distillation for Effective-and-Efficient Vision-and-Language Navigation

arXiv 2406.17960 ↗First posted 25 Jun 2024cs.CV cs.AIcited by 6

On the map: Knowledge distillation → Multimodal distillation

Abstract (arXiv)

Despite the remarkable developments of recent large models in Embodied Artificial Intelligence (E-AI), their integration into robotics is hampered by their excessive parameter sizes and computational demands. Towards the Vision-and-Language Navigation (VLN) task, a core task in E-AI, this paper reveals the great potential of using knowledge distillation for obtaining lightweight student models by proposing a Meta-Ability Guided Interactive Chain-of-distillation (MAGIC) method. Specifically, a Meta-Ability Knowledge Distillation (MAKD) framework is proposed for decoupling and refining the necessary meta-abilities of VLN agents. A Meta-Knowledge Randomization Weighting (MKRW) and a Meta-Knowledge Transferable Determination (MKTD) module are incorporated to dynamically adjust aggregation weights at the meta-ability and sample levels, respectively. Move beyond the traditional one-step unidirectional distillation, an Interactive Chain-of-Distillation (ICoD) learning strategy is proposed to allow students to give feedback to teachers, forming a new multi-step teacher-student co-evolution pipeline. Remarkably, on the R2R test unseen public leaderboard, our smallest model, MAGIC-S, with only 5% (11M) of the teacher's size, outperforms all previous methods under the same training data. Additionally, our largest model, MAGIC-L, surpasses the previous state-of-the-art by 5.84% in SPL and 3.18% in SR. Furthermore, a new dataset was collected and annotated from our living environments, where MAGIC-S demonstrated superior performance and real-time efficiency. Our code is publicly available on https://github.com/CrystalSixone/VLN-MAGIC.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Multimodal distillation, one of 12 directions in Knowledge distillation. Builds on 63 of its 85 references that are in the map. First posted on arXiv 25 Jun 2024.

Builds on

63 of 85 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023DLIP: Distilling Language-Image Pre-training24 Aug
  2. 2022Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation12 Mar

Other references · 61

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  8. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  9. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  10. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  11. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  12. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
38 more
  1. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  2. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  3. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  4. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  5. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  6. 2019Improved Knowledge Distillation via Teacher Assistantcited by 573
  7. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
  8. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  9. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
  10. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  11. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
  12. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  13. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
  14. 2021An Empirical Study of Training End-to-End Vision-and-Language Transformerscited by 271
  15. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 244
  16. 2022TinyViT: Fast Pretraining Distillation for Small Vision Transformerscited by 234
  17. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 231
  18. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
  19. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 130
  20. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
  21. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 122
  22. 2023L3MVN: Leveraging Large Language Models for Visual Target Navigationcited by 120
  23. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
  24. 2018An Alternative View: When Does SGD Escape Local Minima?cited by 115
  25. 2021Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learningcited by 101
  26. 2023TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritancecited by 90
  27. 2020Sim-to-Real Transfer for Vision-and-Language Navigationcited by 89
  28. 2021Causal Attention for Unbiased Visual Recognitioncited by 80
  29. 2020Object-and-Action Aware Model for Visual Language Navigationcited by 77
  30. 2020Sub-Instruction Aware Vision-and-Language Navigationcited by 66
  31. 2021Less is More: Generating Grounded Navigation Instructions from Landmarkscited by 64
  32. 2022Target-Driven Structured Transformer Planner for Vision-Language Navigationcited by 58
  33. 2021SOAT: A Scene- and Object-Aware Transformer for Vision-and-Language Navigationcited by 53
  34. 2024Vision-and-Language Navigation via Causal Learningcited by 51
  35. 2021Neighbor-view Enhanced Model for Vision and Language Navigationcited by 50
  36. 2023Learning Vision-and-Language Navigation from YouTube Videoscited by 47
  37. 2022ADAPT: Vision-Language Navigation with Modality-Aligned Action Promptscited by 45
  38. 2023KERM: Knowledge Enhanced Reasoning for Vision-and-Language Navigationcited by 39

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 4.

6 papers.

  1. 2024Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheelcited by 20
  2. 2024MiniVLN: Efficient Vision-and-Language Navigation by Progressive Knowledge Distillationcited by 6
  3. 2026Trajectory-Diversity-Driven Robust Vision-and-Language Navigationcited by 5
  4. 2026All-day Multi-scenes Lifelong Vision-and-Language Navigation with Tucker Adaptationcited by 2
  5. 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigationcited by 1
  6. 2026Query, Align, and Distill: Navigation-Aware Cross-Modal Interaction for Efficient Vision-and-Language Navigationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning
  2. 2026SymVD: Symmetric Vision Language Action Distillation for Robot Manipulation
  3. 2025SE-VLN: A Self-Evolving Vision-Language Navigation Framework Based on Multimodal Large Language Models
  4. 2025VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluation
  5. 2026Training Generalist Navigation Policies via Action Self-Distillation
  6. 2025Landmark-Guided Knowledge for Vision-and-Language Navigation
  7. 2026From Images to Words: Efficient Cross-Modal Knowledge Distillation to Language Models from Black-box Teachers
  8. 2025Knowledge Distillation with Training Wheels