mlsanityData to 7 Oct 2026

Uncertainty-Aware Gaussian Map for Vision-Language Navigation

arXiv 2605.26503 ↗First posted 26 May 2026cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Vision-Language Navigation (VLN) requires an agent to navigate 3D environments following natural language instructions. During navigation, existing agents commonly encounter perceptual uncertainty, such as insufficient evidence for reliable grounding or ambiguity in interpreting spatial cues, yet they typically ignore such information when predicting actions. In this work, we explicitly model three forms of perceptual uncertainty (i.e., geometric, semantic, and appearance uncertainty) and integrate them into the agent's observation space to enable informed decision-making. Concretely, our agent first constructs a Semantic Gaussian Map (SGM), composed of differentiable 3D Gaussian primitives initialized from panoramic observations, that encodes both the geometric structure and semantic content of the environment. On top of SGM, geometric uncertainty is estimated through variational perturbations of Gaussian position and scale to assess structural reliability; semantic uncertainty is captured by perturbing Gaussian semantic attributes to reveal ambiguous interpretations; and appearance uncertainty is characterized by Fisher Information, which measures the sensitivity of rendered observations to Gaussian-level variations. These uncertainties are incorporated into SGM, extending it into a unified 3D Value Map, which grounds them as affordances and constraints that support reliable navigation. Comprehensive evaluations across multiple VLN benchmarks show the effectiveness of our agent.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 76 of its 98 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

76 of 98 references.

References · 76

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  4. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  7. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  8. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  9. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  10. 2023Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstructioncited by 753
  11. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  12. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
38 more
  1. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  2. 2023SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAMcited by 386
  3. 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 345
  4. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  5. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  6. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  7. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  8. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  9. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  10. 2024Navigation World Modelscited by 262
  11. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  12. 2020Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learningcited by 208
  13. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  14. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  15. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 129
  16. 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 123
  17. 2021Pathdreamer: A World Model for Indoor Navigationcited by 117
  18. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  19. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
  20. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 117
  21. 2020Language and Visual Entity Relationship Graph for Agent Navigationcited by 100
  22. 2021Structured Scene Memory for Vision-Language Navigationcited by 95
  23. 2023Bayes' Rays: Uncertainty Quantification for Neural Radiance Fieldscited by 90
  24. 2024PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splattingcited by 85
  25. 2018Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigationcited by 84
  26. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  27. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  28. 2021Stochastic Neural Radiance Fields: Quantifying Uncertainty in Implicit 3D Representationscited by 80
  29. 2024Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Taskscited by 78
  30. 2020Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigationcited by 71
  31. 2022Conditional-Flow NeRF: Accurate 3D Modelling with Reliable Uncertainty Quantificationcited by 71
  32. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 71
  33. 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 69
  34. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  35. 2022Target-Driven Structured Transformer Planner for Vision-Language Navigationcited by 58
  36. 2022Density-aware NeRF Ensembles: Quantifying Predictive Uncertainty in Neural Radiance Fieldscited by 58
  37. 2020Active Visual Information Gathering for Vision-Language Navigationcited by 55
  38. 2023Open-Fusion: Real-time Open-Vocabulary 3D Mapping and Queryable Scene Representationcited by 54

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Uncertainty-Aware World Model for Aerial Image-Goal Navigationcited by 2