mlsanityData to 7 Oct 2026

Vision-and-Language Navigation via Causal Learning

arXiv 2404.10241 ↗First posted 16 Apr 2024cs.CV cs.AIcited by 51

Not on the map. Why?

Abstract (arXiv)

In the pursuit of robust and generalizable environment perception and language understanding, the ubiquitous challenge of dataset bias continues to plague vision-and-language navigation (VLN) agents, hindering their performance in unseen environments. This paper introduces the generalized cross-modal causal transformer (GOAT), a pioneering solution rooted in the paradigm of causal inference. By delving into both observable and unobservable confounders within vision, language, and history, we propose the back-door and front-door adjustment causal learning (BACL and FACL) modules to promote unbiased learning by comprehensively mitigating potential spurious correlations. Additionally, to capture global confounder features, we propose a cross-modal feature pooling (CFP) module supervised by contrastive learning, which is also shown to be effective in improving cross-modal representations during pre-training. Extensive experiments across multiple VLN datasets (R2R, REVERIE, RxR, and SOON) underscore the superiority of our proposed method over previous state-of-the-art approaches. Code is available at https://github.com/CrystalSixone/VLN-GOAT.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). Builds on 66 of its 80 references that are in the map. First posted on arXiv 16 Apr 2024.

Builds on

66 of 80 references.

References · 66

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2016Layer Normalizationcited by 5,579
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  8. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  9. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  10. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  11. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  12. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
38 more
  1. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  2. 2021An Empirical Study of Training End-to-End Vision-and-Language Transformerscited by 271
  3. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  4. 2020Counterfactual VQA: A Cause-Effect Look at Language Biascited by 266
  5. 2019Vision-and-Dialog Navigationcited by 246
  6. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  7. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
  8. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  9. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  10. 2019Stay on the Path: Instruction Fidelity in Vision-and-Language Navigationcited by 129
  11. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 129
  12. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 118
  13. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  14. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
  15. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  16. 2020Interventional Few-Shot Learningcited by 112
  17. 2021Causal Attention for Vision-Language Taskscited by 111
  18. 2020Visual Commonsense R-CNNcited by 106
  19. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  20. 2021Causal Attention for Unbiased Visual Recognitioncited by 80
  21. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 71
  22. 2021Vision-Language Navigation with Random Environmental Mixupcited by 70
  23. 2020Deconfounded Image Captioning: A Causal Retrospectcited by 65
  24. 2021Less is More: Generating Grounded Navigation Instructions from Landmarkscited by 62
  25. 2022A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learningcited by 60
  26. 2022Target-Driven Structured Transformer Planner for Vision-Language Navigationcited by 58
  27. 2022Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answeringcited by 57
  28. 2019Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigationcited by 54
  29. 2021SOAT: A Scene- and Object-Aware Transformer for Vision-and-Language Navigationcited by 53
  30. 2022From Representation to Reasoning: Towards both Evidence and Commonsense Reasoning for Video Question-Answeringcited by 51
  31. 2021Neighbor-view Enhanced Model for Vision and Language Navigationcited by 50
  32. 2022Counterfactual Cycle-Consistent Learning for Instruction Following and Generation in Vision-Language Navigationcited by 50
  33. 2021Scene-Intuitive Agent for Remote Embodied Visual Groundingcited by 49
  34. 2023Learning Vision-and-Language Navigation from YouTube Videoscited by 47
  35. 20223D-Aware Object Goal Navigation via Simultaneous Exploration and Identificationcited by 46
  36. 2023Improving Vision-and-Language Navigation by Generating Future-View Image Semanticscited by 46
  37. 2022ADAPT: Vision-Language Navigation with Modality-Aligned Action Promptscited by 45
  38. 2020DeVLBert: Learning Deconfounded Visio-Linguistic Representationscited by 42

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 27; 2026: 18.

51 papers.

Sort:

By area:

  1. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  2. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 41
  3. 2025Rethinking the Embodied Gap in Vision-and-Language Navigation: A Holistic Study of Physical and Visual Disparitiescited by 24
  4. 2024RoomTour3D: Geometry-Aware Video-Instruction Tuning for Embodied Navigationcited by 23
  5. 2025Dynam3D: Dynamic Layered 3D Tokens Empower VLM for Vision-and-Language Navigationcited by 21
  6. 2024g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Taskscited by 18
  7. 2024Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheelcited by 18
  8. 2025NavRAG: Generating User Demand Instructions for Embodied Navigation through Retrieval-Augmented LLMcited by 18
  9. 2025Aerial Vision-and-Language Navigation with Grid-based View Selection and Map Constructioncited by 12
  10. 2025COSMO: Combination of Selective Memorization for Low-cost Vision-and-Language Navigationcited by 12
  11. 2025CleanPose: Category-Level Object Pose Estimation via Causal Learning and Knowledge Distillationcited by 9
  12. 2025CAUSAL3D: A Comprehensive Benchmark for Causal Learning from Visual Datacited by 7
38 more
  1. 2025Active Test-time Vision-Language Navigationcited by 7
  2. 2025VLA-4D: Embedding 4D Awareness into Vision-Language-Action Models for SpatioTemporally Coherent Robotic Manipulationcited by 7
  3. 2024MAGIC: Meta-Ability Guided Interactive Chain-of-Distillation for Effective-and-Efficient Vision-and-Language Navigationcited by 6
  4. 2024MiniVLN: Efficient Vision-and-Language Navigation by Progressive Knowledge Distillationcited by 6
  5. 2025PanoGen++: Domain-Adapted Text-Guided Panoramic Environment Generation for Vision-and-Language Navigationcited by 6
  6. 2025NavQ: Learning a Q-Model for Foresighted Vision-and-Language Navigationcited by 6
  7. 2025CoNav: Collaborative Cross-Modal Reasoning for Embodied Navigationcited by 5
  8. 2025NavComposer: Composing Language Instructions for Navigation Trajectories through Action-Scene-Object Modularizationcited by 5
  9. 2025[De|Re]constructing VLMs' Reasoning in Countingcited by 5
  10. 2025Weakly-supervised VLM-guided Partial Contrastive Learning for Visual Language Navigationcited by 4
  11. 2025Implicit Counterfactual Learning for Audio-Visual Segmentationcited by 4
  12. 2025Boosting Zero-Shot VLN via Abstract Obstacle Map-Based Waypoint Prediction with TopoGraph-and-VisitInfo-Aware Promptingcited by 4
  13. 2026VLM-Loc: Localization in Point Cloud Maps via Vision-Language Modelscited by 4
  14. 2026Trajectory-Diversity-Driven Robust Vision-and-Language Navigationcited by 4
  15. 2026Instruction-as-State: Environment-Guided and State-Conditioned Semantic Understanding for Embodied Navigationcited by 4
  16. 2025Revealing Multimodal Causality with Large Language Modelscited by 3
  17. 2025Expand Your SCOPE: Semantic Cognition over Potential-Based Exploration for Embodied Visual Navigationcited by 3
  18. 2025Empowering Vision Transformers with Multi-Scale Causal Intervention for Long-Tailed Image Classificationcited by 2
  19. 2026Deconfounded Lifelong Learning for Autonomous Driving via Dynamic Knowledge Spacescited by 2
  20. 2026P2DNav: Panorama-to-Downview Reasoning for Zero-shot Vision-and-Language Navigationcited by 2
  21. 2026Ask When It Pays: Cost-Aware Open-Ended Interaction for Instance Goal Navigationcited by 2
  22. 2025Cross-modal Causal Intervention for Alzheimer's Disease Predictioncited by 1
  23. 2025Robust Egocentric Referring Video Object Segmentation via Dual-Modal Causal Interventioncited by 1
  24. 2026Mitigating Error Accumulation in Continuous Navigation via Memory-Augmented Kalman Filteringcited by 1
  25. 2026The Essence of Balance for Self-Improving Agents in Vision-and-Language Navigationcited by 1
  26. 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigationcited by 1
  27. 2026OmniEcho: Audio-Visual Spatial Understanding for Omni-Modal Embodied Agentscited by 1
  28. 2025ST-Booster: An Iterative SpatioTemporal Perception Booster for Vision-and-Language Navigation in Continuous Environmentscited by 0
  29. 2025Causality-Driven Infrared and Visible Image Fusioncited by 0
  30. 2025DISCO: Mitigating Bias in Deep Learning with Conditional Distance Correlationcited by 0
  31. 2025Walk and Read Less: Improving the Efficiency of Vision-and-Language Navigation via Tuning-Free Multimodal Token Pruningcited by 0
  32. 2026Global Commander and Local Operative: A Dual-Agent Framework for Scene Navigationcited by 0
  33. 2026Enhancing Vision-Language Navigation with Multimodal Event Knowledge from Real-World Indoor Tour Videoscited by 0
  34. 2026Implicit Geometry Representations for Vision-and-Language Navigation from Web Videoscited by 0
  35. 2026Beyond Textual Knowledge-Leveraging Multimodal Knowledge Bases for Enhancing Vision-and-Language Navigationcited by 0
  36. 2026Bridge: Basis-Driven Causal Inference Marries VFMs for Domain Generalizationcited by 0
  37. 2026TriRelVLA: Triadic Relational Structure for Generalizable Embodied Manipulationcited by 0
  38. 2026Advancing DialNav through Automatic Embodied Dialog Augmentationcited by 0

50 most cited of 51 All 51 (JSON)