mlsanityData to 7 Oct 2026

Self-Monitoring Navigation Agent via Auxiliary Progress Estimation

arXiv 1901.03035 ↗First posted 10 Jan 2019cs.AI cs.CL cs.CVcited by 173

Not on the map. Why?

Abstract (arXiv)

The Vision-and-Language Navigation (VLN) task entails an agent following navigational instruction in photo-realistic unknown environments. This challenging task demands that the agent be aware of which instruction was completed, which instruction is needed next, which way to go, and its navigation progress towards the goal. In this paper, we introduce a self-monitoring agent with two complementary components: (1) visual-textual co-grounding module to locate the instruction completed in the past, the instruction required for the next action, and the next moving direction from surrounding images and (2) progress monitor to ensure the grounded instruction correctly reflects the navigation progress. We test our self-monitoring agent on a standard benchmark and analyze our proposed approach through a series of ablation studies that elucidate the contributions of the primary components. Using our proposed method, we set the new state of the art by a significant margin (8% absolute increase in success rate on the unseen test set). Code is available at https://github.com/chihyaoma/selfmonitoring-agent .

Key facts. Cited by 173 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 38; 2022: 35; 2023: 33; 2024: 30; 2025: 18; 2026: 18.

173 papers.

Sort:

By area:

  1. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  2. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
  3. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  4. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  5. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  6. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  7. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 167
  8. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  9. 2021Episodic Transformer for Vision-and-Language Navigationcited by 155
  10. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 130
  11. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 129
  12. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
38 more
  1. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 118
  2. 2021Pathdreamer: A World Model for Indoor Navigationcited by 117
  3. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  4. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  5. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  6. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  7. 2021Skill Induction and Planning with Latent Languagecited by 99
  8. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 96
  9. 2021Structured Scene Memory for Vision-Language Navigationcited by 95
  10. 2021A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Executioncited by 91
  11. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  12. 2021LanguageRefer: Spatial-Language Model for 3D Visual Groundingcited by 88
  13. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 87
  14. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  15. 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 85
  16. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  17. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  18. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  19. 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
  20. 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 77
  21. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 71
  22. 2021Vision-Language Navigation with Random Environmental Mixupcited by 70
  23. 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 69
  24. 2023A2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Modelscited by 61
  25. 2021The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigationcited by 60
  26. 2022A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learningcited by 60
  27. 2024Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigationcited by 60
  28. 2022Target-Driven Structured Transformer Planner for Vision-Language Navigationcited by 58
  29. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  30. 2021Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoringcited by 53
  31. 2021SOAT: A Scene- and Object-Aware Transformer for Vision-and-Language Navigationcited by 53
  32. 2021RobustNav: Towards Benchmarking Robustness in Embodied Navigationcited by 52
  33. 2021Neighbor-view Enhanced Model for Vision and Language Navigationcited by 50
  34. 2022Counterfactual Cycle-Consistent Learning for Instruction Following and Generation in Vision-Language Navigationcited by 50
  35. 2021Scene-Intuitive Agent for Remote Embodied Visual Groundingcited by 49
  36. 2022Learning from Unlabeled 3D Environments for Vision-and-Language Navigationcited by 49
  37. 2024Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environmentscited by 49
  38. 2021SASRA: Semantically-aware Spatio-temporal Reasoning Agent for Vision-and-Language Navigation in Continuous Environmentscited by 48

50 most cited of 173 All 173 (JSON)