mlsanityData to 7 Oct 2026

GSMem: 3D Gaussian Splatting as Persistent Spatial Memory for Zero-Shot Embodied Exploration and Reasoning

arXiv 2603.19137 ↗First posted 19 Mar 2026cs.CV cs.ROcited by 5

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Effective embodied exploration requires agents to accumulate and retain spatial knowledge over time. However, existing scene representations, such as discrete scene graphs or static view-based snapshots, lack textitpost-hoc re-observability. If an initial observation misses a target, the resulting memory omission is often irrecoverable. To bridge this gap, we propose GSMem, a zero-shot embodied exploration and reasoning framework built upon 3D Gaussian Splatting (3DGS). By explicitly parameterizing continuous geometry and dense appearance, 3DGS serves as a persistent spatial memory that endows the agent with textitSpatial Recollection: the ability to render photorealistic novel views from optimal, previously unoccupied viewpoints. To operationalize this, GSMem employs a retrieval mechanism that simultaneously leverages parallel object-level scene graphs and semantic-level language fields. This complementary design robustly localizes target regions, enabling the agent to ``hallucinate'' optimal views for high-fidelity Vision-Language Model (VLM) reasoning. Furthermore, we introduce a hybrid exploration strategy that combines VLM-driven semantic scoring with a 3DGS-based coverage objective, balancing task-aware exploration with geometric coverage. Extensive experiments on embodied question answering and lifelong navigation demonstrate the robustness and effectiveness of our framework

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 34 references that are in the map. First posted on arXiv 19 Mar 2026.

Builds on

29 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answering8 Nov

Other references · 28

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  5. 2024GPT-4o System Cardcited by 4,467
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  7. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  8. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  9. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
  10. 2023LangSplat: 3D Language Gaussian Splattingcited by 410
  11. 2023Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fieldscited by 361
  12. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
16 more
  1. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 201
  2. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  3. 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Modelscited by 160
  4. 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 106
  5. 2023SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentationcited by 95
  6. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
  7. 2023GOAT: GO to Any Thingcited by 89
  8. 2024Explore until Confident: Efficient Exploration for Embodied Question Answeringcited by 74
  9. 20243D-Mem: 3D Scene Memory for Embodied Exploration and Reasoningcited by 73
  10. 2023FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Informationcited by 63
  11. 2023Foundations of Spatial Perception for Robotics: Hierarchical Representations and Real-time Systemscited by 60
  12. 2025Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigationcited by 50
  13. 2024TANGO: Training-free Embodied AI Agents for Open-world Taskscited by 26
  14. 2024Occam's LGS: An Efficient Approach for Language Gaussian Splattingcited by 15
  15. 2025Online Language Splattingcited by 10
  16. 2025DyNaVLM: Zero-Shot Vision-Language Navigation System with Dynamic Viewpoints and Self-Refining Graph Memorycited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026RenderMem: Rendering as Spatial Memory Retrieval
  2. 20263DGSNav: Enhancing Vision-Language Model Reasoning for Object Navigation via Active 3D Gaussian Splatting
  3. 2026SplatReasoner: Enhancing Embodied Reasoning and Grounding by Novel View Synthesis
  4. 2025Vision to Geometry: 3D Spatial Memory for Sequential Embodied MLLM Reasoning and Exploration
  5. 2026E3VS-Bench: A Benchmark for Viewpoint-Dependent Active Perception in 3D Gaussian Splatting Scenes
  6. 2026RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics
  7. 2026EvoMemNav: Efficient Self-Evolving Fine-Grained Memory for Zero-Shot Embodied Navigation
  8. 2026SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments