mlsanityData to 14 Oct 2026

ST-Booster: An Iterative SpatioTemporal Perception Booster for Vision-and-Language Navigation in Continuous Environments

arXiv 2504.09843 ↗First posted 14 Apr 2025cs.CV cs.ROcited by 0

Not on the map. Why?

Abstract (arXiv)

Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires agents to navigate unknown, continuous spaces based on natural language instructions. Compared to discrete settings, VLN-CE poses two core perception challenges. First, the absence of predefined observation points leads to heterogeneous visual memories and weakened global spatial correlations. Second, cumulative reconstruction errors in three-dimensional scenes introduce structural noise, impairing local feature perception. To address these challenges, this paper proposes ST-Booster, an iterative spatiotemporal booster that enhances navigation performance through multi-granularity perception and instruction-aware reasoning. ST-Booster consists of three key modules -- Hierarchical SpatioTemporal Encoding (HSTE), Multi-Granularity Aligned Fusion (MGAF), and ValueGuided Waypoint Generation (VGWG). HSTE encodes long-term global memory using topological graphs and captures shortterm local details via grid maps. MGAF aligns these dualmap representations with instructions through geometry-aware knowledge fusion. The resulting representations are iteratively refined through pretraining tasks. During reasoning, VGWG generates Guided Attention Heatmaps (GAHs) to explicitly model environment-instruction relevance and optimize waypoint selection. Extensive comparative experiments and performance analyses are conducted, demonstrating that ST-Booster outperforms existing state-of-the-art methods, particularly in complex, disturbance-prone environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 54 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

38 of 54 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  5. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  6. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  7. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  8. 2022Visual Language Maps for Robot Navigationcited by 365
  9. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
  10. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
  11. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
  12. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 132
26 more
  1. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 129
  2. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
  3. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 122
  4. 2021FILM: Following Instructions in Language with Modular Methodscited by 121
  5. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
  6. 2020Topological Planning with Transformers for Vision-and-Language Navigationcited by 116
  7. 2022Navigating to Objects in the Real Worldcited by 106
  8. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 94
  9. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  10. 2020Sim-to-Real Transfer for Vision-and-Language Navigationcited by 89
  11. 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 86
  12. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  13. 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 86
  14. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 85
  15. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 72
  16. 2024Vision-and-Language Navigation via Causal Learningcited by 51
  17. 2021Neighbor-view Enhanced Model for Vision and Language Navigationcited by 50
  18. 2021SASRA: Semantically-aware Spatio-temporal Reasoning Agent for Vision-and-Language Navigation in Continuous Environmentscited by 48
  19. 20223D-Aware Object Goal Navigation via Simultaneous Exploration and Identificationcited by 46
  20. 2023Improving Vision-and-Language Navigation by Generating Future-View Image Semanticscited by 46
  21. 2023Learning Navigational Visual Representations with Semantic Map Supervisioncited by 43
  22. 2023GeoVLN: Learning Geometry-Enhanced Visual Representation with Slot Attention for Vision-and-Language Navigationcited by 24
  23. 2024Navigating Beyond Instructions: Vision-and-Language Navigation in Obstructed Environmentscited by 10
  24. 2022The Foreseeable Future: Self-Supervised Learning to Predict Dynamic Scenes for Indoor Navigationcited by 5
  25. 2024BEVNav: Robot Autonomous Navigation Via Spatial-Temporal Contrastive Learning in Bird's-Eye Viewcited by 3
  26. 2024Real-Time Metric-Semantic Mapping for Autonomous Navigation in Outdoor Environmentscited by 2

Built on by

Not cited yet.