mlsanityData to 14 Oct 2026

Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigation

arXiv 2311.13209 ↗First posted 22 Nov 2023cs.CVcited by 25

Not on the map. Why?

Abstract (arXiv)

The ability to accurately comprehend natural language instructions and navigate to the target location is essential for an embodied agent. Such agents are typically required to execute user instructions in an online manner, leading us to explore the use of unlabeled test samples for effective online model adaptation. However, for online Vision-and-Language Navigation (VLN), due to the intrinsic nature of inter-sample online instruction execution and intra-sample multi-step action decision, frequent updates can result in drastic changes in model parameters, while occasional updates can make the model ill-equipped to handle dynamically changing environments. Therefore, we propose a Fast-Slow Test-Time Adaptation (FSTTA) approach for online VLN by performing joint decomposition-accumulation analysis for both gradients and parameters in a unified framework. Extensive experiments show that our method obtains impressive performance gains on four popular benchmarks. Code is available at https://github.com/Feliciaxyao/ICML2024-FSTTA.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 83 of its 97 references that are in the map. First posted on arXiv 22 Nov 2023.

Builds on

83 of 97 references.

References · 83

  1. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  2. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  3. 2022Continual Test-Time Domain Adaptationcited by 525
  4. 2022Efficient Test-Time Model Adaptation without Forgettingcited by 469
  5. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  6. 2023Towards Stable Test-Time Adaptation in Dynamic Wild Worldcited by 415
  7. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  8. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 377
  9. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 359
  10. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
  11. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 333
  12. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
38 more
  1. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
  2. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
  3. 2021Gradient Matching for Domain Generalizationcited by 258
  4. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 244
  5. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 231
  6. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
  7. 2022Parameter-free Online Test-time Adaptationcited by 189
  8. 2023Robust Test-Time Adaptation in Dynamic Scenarioscited by 189
  9. 2021Fishr: Invariant Gradient Variances for Out-of-Distribution Generalizationcited by 186
  10. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 179
  11. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 176
  12. 2022NOTE: Robust Continual Test-time Adaptation Against Temporal Correlationcited by 151
  13. 2014A Tutorial on Principal Component Analysiscited by 141
  14. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 132
  15. 2021The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalizationcited by 131
  16. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 130
  17. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 129
  18. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
  19. 2023EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularizationcited by 123
  20. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 119
  21. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 117
  22. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 117
  23. 2020Topological Planning with Transformers for Vision-and-Language Navigationcited by 116
  24. 2019The Regretful Agent: Heuristic-Aided Navigation through Progress Estimationcited by 105
  25. 2022Robust Mean Teacher for Continual and Gradual Test-Time Adaptationcited by 104
  26. 2023Sharpness-Aware Gradient Matching for Domain Generalizationcited by 102
  27. 2023TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptationcited by 101
  28. 2018Adapting Auxiliary Losses Using Gradient Similaritycited by 100
  29. 2020Language and Visual Entity Relationship Graph for Agent Navigationcited by 100
  30. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 94
  31. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  32. 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 86
  33. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  34. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 85
  35. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  36. 2019Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigationcited by 81
  37. 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
  38. 2019Robust Navigation with Language Pretraining and Stochastic Samplingcited by 77

50 most cited of 83 All 83 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 10; 2026: 13.

25 papers.

Sort:

By area:

  1. 2024Navigation World Modelscited by 272
  2. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  3. 2025NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environmentscited by 45
  4. 2025Centaur: Robust End-to-End Autonomous Driving with Test-Time Trainingcited by 42
  5. 2025Beyond Entropy: Region Confidence Proxy for Wild Test-Time Adaptationcited by 11
  6. 2025Neural Brain: A Neuroscience-inspired Framework for Embodied Agentscited by 10
  7. 2025Active Test-time Vision-Language Navigationcited by 7
  8. 2025NavQ: Learning a Q-Model for Foresighted Vision-and-Language Navigationcited by 6
  9. 2026MapDream: Task-Driven Map Learning for Vision-Language Navigationcited by 4
  10. 2025Run, Ruminate, and Regulate: A Dual-process Thinking System for Vision-and-Language Navigationcited by 2
  11. 2026All-day Multi-scenes Lifelong Vision-and-Language Navigation with Tucker Adaptationcited by 2
  12. 2026Dual-level Adaptation for Multi-Object Tracking: Building Test-Time Calibration from Experience and Intuitioncited by 2
13 more
  1. 2026Turning Adaptation into Assets: Cross-Domain Bridging for Online Vision-Language Navigationcited by 2
  2. 2026OmniEcho: Audio-Visual Spatial Understanding for Omni-Modal Embodied Agentscited by 2
  3. 2025Unleashing the Potential of All Test Samples: Mean-Shift Guided Test-Time Adaptationcited by 1
  4. 2025Adaptive Dual Uncertainty Optimization: Boosting Monocular 3D Object Detection under Test-Time Shiftscited by 1
  5. 2025User-Feedback-Driven Adaptation for Vision-and-Language Navigationcited by 1
  6. 2026Mitigating Error Accumulation in Continuous Navigation via Memory-Augmented Kalman Filteringcited by 1
  7. 2026SpatialFly: Implicit 3D Prior-Guided Visual Reparameterization for Continuous UAV Vision-and-Language Navigationcited by 1
  8. 2026Global Commander and Local Operative: A Dual-Agent Framework for Scene Navigationcited by 0
  9. 2026NavOne: One-Step Global Planning for Vision-Language Navigation on Top-Down Mapscited by 0
  10. 2026Fisher-Preserving Guidance: Training-Free Manifold Constraints for Safe Diffusion Controlcited by 0
  11. 2026FlowDec: Temporal Conditional Flow Decorruptor for Robust Continuous Vision-Language Navigationcited by 0
  12. 2026Towards Dual-Brain Minimal Sufficient Representation for Vision-Language Navigationcited by 0
  13. 2026Credit-Guided Policy Improvement for Test-time Adaptive Vision-Language Navigationcited by 0