mlsanityData to 14 Oct 2026

EnvEdit: Environment Editing for Vision-and-Language Navigation

arXiv 2203.15685 ↗First posted 29 Mar 2022cs.CV cs.AI cs.CLcited by 83

Not on the map. Why?

Abstract (arXiv)

In Vision-and-Language Navigation (VLN), an agent needs to navigate through the environment based on natural language instructions. Due to limited available data for agent training and finite diversity in navigation environments, it is challenging for the agent to generalize to new, unseen environments. To address this problem, we propose EnvEdit, a data augmentation method that creates new environments by editing existing environments, which are used to train a more generalizable agent. Our augmented environments can differ from the seen environments in three diverse aspects: style, object appearance, and object classes. Training on these edit-augmented environments prevents the agent from overfitting to existing environments and helps generalize better to new, unseen environments. Empirically, on both the Room-to-Room and the multi-lingual Room-Across-Room datasets, we show that our proposed EnvEdit method gets significant improvements in all metrics on both pre-trained and non-pre-trained VLN agents, and achieves the new state-of-the-art on the test leaderboard. We further ensemble the VLN agents augmented on different edited environments and show that these edit methods are complementary. Code and data are available at https://github.com/jialuli-luka/EnvEdit

Key facts. Cited by 83 papers in the map (data through 14 Oct 2026). Builds on 68 of its 81 references that are in the map. First posted on arXiv 29 Mar 2022.

Builds on

68 of 81 references.

References · 68

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  5. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  6. 2022Generative Adversarial Networkscited by 5,627
  7. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  8. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  9. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  10. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  11. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  12. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
38 more
  1. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  2. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  3. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  4. 2018On Evaluation of Embodied Navigation Agentscited by 547
  5. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  6. 2017The Effectiveness of Data Augmentation in Image Classification using Deep Learningcited by 370
  7. 2016A Learned Representation For Artistic Stylecited by 341
  8. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
  9. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  10. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
  11. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
  12. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
  13. 2019Vision-and-Dialog Navigationcited by 250
  14. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 244
  15. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
  16. 2017Geometric GANcited by 233
  17. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 231
  18. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 179
  19. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  20. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 176
  21. 2019ManiGAN: Text-Guided Image Manipulationcited by 156
  22. 2018Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networkscited by 147
  23. 2019Stay on the Path: Instruction Fidelity in Vision-and-Language Navigationcited by 130
  24. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 130
  25. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 119
  26. 2021Pathdreamer: A World Model for Indoor Navigationcited by 118
  27. 2018GAN Augmentation: Augmenting Training Data using Generative Adversarial Networkscited by 107
  28. 2021Paint by Wordcited by 107
  29. 2019General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warpingcited by 105
  30. 2018Mapping Instructions to Actions in 3D Environments with Visual Goal Predictioncited by 101
  31. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  32. 2020Language and Visual Entity Relationship Graph for Agent Navigationcited by 100
  33. 2018Data Augmentation by Pairing Samples for Images Classificationcited by 96
  34. 2018Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Languagecited by 90
  35. 2017Exploring the structure of a real-time, arbitrary neural artistic stylization networkcited by 89
  36. 2017Semantic Image Synthesis via Adversarial Learningcited by 87
  37. 2018Style Augmentation: Data Augmentation via Style Randomizationcited by 79
  38. 2021Vision-Language Navigation with Random Environmental Mixupcited by 70

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 8; 2023: 23; 2024: 25; 2025: 16; 2026: 11.

83 papers.

Sort:

By area:

  1. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  2. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
  3. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
  4. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 122
  5. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
  6. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 110
  7. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 98
  8. 2022SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learningcited by 95
  9. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  10. 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
  11. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 72
  12. 2023OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNavcited by 70
38 more
  1. 2023VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street Viewcited by 67
  2. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  3. 2024Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMscited by 62
  4. 2022A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learningcited by 60
  5. 2022CLIP-Nav: Using CLIP for Zero-Shot Vision-and-Language Navigationcited by 55
  6. 2024Vision-and-Language Navigation via Causal Learningcited by 51
  7. 2022Learning from Unlabeled 3D Environments for Vision-and-Language Navigationcited by 50
  8. 2024Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environmentscited by 50
  9. 2023Learning Vision-and-Language Navigation from YouTube Videoscited by 47
  10. 2023Improving Vision-and-Language Navigation by Generating Future-View Image Semanticscited by 46
  11. 2024EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agentscited by 46
  12. 2023Learning Navigational Visual Representations with Semantic Map Supervisioncited by 43
  13. 2025Multimodal Fusion and Vision-Language Models: A Survey for Robot Visioncited by 40
  14. 20221st Place Solutions for RxR-Habitat Vision-and-Language Navigation Competition (CVPR 2022)cited by 34
  15. 2023Grounded Entity-Landmark Adaptive Pre-training for Vision-and-Language Navigationcited by 27
  16. 2023Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigationcited by 25
  17. 2024DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedbackcited by 25
  18. 2023VLN-PETL: Parameter-Efficient Transfer Learning for Vision-and-Language Navigationcited by 23
  19. 2024Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheelcited by 20
  20. 2024Human-Aware Vision-and-Language Navigation: Bridging Simulation to Reality with Dynamic Human Interactionscited by 19
  21. 2024Vision-Language Navigation with Energy-Based Policycited by 19
  22. 2023A Dual Semantic-Aware Recurrent Global-Adaptive Network For Vision-and-Language Navigationcited by 18
  23. 2025FlexVLN: Flexible Adaptation for Diverse Vision-and-Language Navigation Taskscited by 17
  24. 2025Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigationcited by 17
  25. 2024SAME: Learning Generic Language-Guided Visual Navigation with State-Adaptive Mixture of Expertscited by 16
  26. 2025COSMO: Combination of Selective Memorization for Low-cost Vision-and-Language Navigationcited by 15
  27. 2024VLN-Video: Utilizing Driving Videos for Outdoor Vision-and-Language Navigationcited by 14
  28. 2024NavHint: Vision and Language Navigation Agent with a Hint Generatorcited by 12
  29. 2022CLEAR: Improving Vision-Language Navigation with Cross-Lingual, Environment-Agnostic Representationscited by 10
  30. 2023ESceme: Vision-and-Language Navigation with Episodic Scene Memorycited by 10
  31. 2023Graph based Environment Representation for Vision-and-Language Navigation in Continuous Environmentscited by 9
  32. 2024Why Only Text: Empowering Vision-and-Language Navigation with Multi-modal Promptscited by 9
  33. 2023PASTS: Progress-Aware Spatio-Temporal Transformer Speaker For Vision-and-Language Navigationcited by 8
  34. 2023Behavioral Analysis of Vision-and-Language Navigation Agentscited by 8
  35. 2025Unseen from Seen: Rewriting Observation-Instruction Using Foundation Models for Augmenting Vision-Language Navigationcited by 8
  36. 2024DELAN: Dual-Level Alignment for Vision-and-Language Navigation by Cross-Modal Contrastive Learningcited by 7
  37. 2024PRET: Planning with Directed Fidelity Trajectory for Vision and Language Navigationcited by 7
  38. 2025HA-VLN 2.0: An Open Benchmark and Leaderboard for Human-Aware Navigation in Discrete and Continuous Environments with Dynamic Multi-Human Interactionscited by 7

50 most cited of 83 All 83 (JSON)