mlsanityData to 7 Oct 2026

Multi-Scale Gaussian-Language Map for Zero-shot Embodied Navigation and Reasoning

arXiv 2605.01736 ↗First posted 3 May 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Understanding the geometric and semantic structure of environments is essential for embodied navigation and reasoning. Existing semantic mapping methods trade off between explicit geometry and multi-scale semantics, and lack a native interface for large models, thus requiring additional training of feature projection for semantic alignment. To this end, we propose the multi-scale Gaussian-Language Map (GLMap), which introduces three key designs: (1) explicit geometry, (2) multi-scale semantics covering both instance and region concepts, and (3) a dual-modality interface where each semantic unit jointly stores a natural language description and a 3D Gaussian representation. The 3D Gaussians enable compact storage and fast rendering of task-relevant images via Gaussian splatting. To enable efficient incremental construction, we further propose a Gaussian Estimator that analytically derives Gaussian parameters from dense point clouds without gradient-based optimization. Experiments on ObjectNav, InstNav, and SQA tasks show that GLMap effectively enhances target navigation and contextual reasoning, while remaining compatible with large-model-based methods in a zero-shot manner. The code is available at https://github.com/sx-zhang/GLMap.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 58 references that are in the map. First posted on arXiv 3 May 2026.

Builds on

42 of 58 references.

References · 42

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  7. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  8. 2021Grounded Language-Image Pre-trainingcited by 1,147
  9. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  10. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  11. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  12. 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 345
30 more
  1. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 326
  2. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  3. 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 255
  4. 2023Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesiscited by 232
  5. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planningcited by 206
  6. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 201
  7. 2020Neural Topological SLAM for Visual Navigationcited by 193
  8. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 177
  9. 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 170
  10. 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
  11. 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understandingcited by 131
  12. 2023Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenescited by 125
  13. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
  14. 2024Pixel-GS: Density Control with Pixel-aware Gradient for 3D Gaussian Splattingcited by 111
  15. 2022Habitat-Matterport 3D Semantics Datasetcited by 109
  16. 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 106
  17. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Modelscited by 106
  18. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
  19. 2025UniGoal: Towards Universal Zero-shot Goal-oriented Navigationcited by 56
  20. 2024VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Modelcited by 55
  21. 2023How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontierscited by 53
  22. 2024Prioritized Semantic Learning for Zero-shot Instance Navigationcited by 30
  23. 2025ApexNav: An Adaptive Exploration Strategy for Zero-Shot Object Navigation with Target-centric Semantic Fusioncited by 29
  24. 20243DGraphLLM: Combining Semantic Graphs and Large Language Models for 3D Scene Understandingcited by 26
  25. 2024GAMap: Zero-Shot Object Goal Navigation with Multi-Scale Geometric-Affordance Guidancecited by 22
  26. 2025Dynam3D: Dynamic Layered 3D Tokens Empower VLM for Vision-and-Language Navigationcited by 21
  27. 2024g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Taskscited by 18
  28. 2025BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigationcited by 18
  29. 2024TopV-Nav: Unlocking the Top-View Spatial Reasoning Potential of MLLM for Zero-shot Object Navigationcited by 15
  30. 2025Distilling LLM Prior to Flow Model for Generalizable Agent's Imagination in Object Goal Navigationcited by 3

Built on by

Not cited yet.