GSMem: 3D Gaussian Splatting as Persistent Spatial Memory for Zero-Shot Embodied Exploration and Reasoning
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Effective embodied exploration requires agents to accumulate and retain spatial knowledge over time. However, existing scene representations, such as discrete scene graphs or static view-based snapshots, lack textitpost-hoc re-observability. If an initial observation misses a target, the resulting memory omission is often irrecoverable. To bridge this gap, we propose GSMem, a zero-shot embodied exploration and reasoning framework built upon 3D Gaussian Splatting (3DGS). By explicitly parameterizing continuous geometry and dense appearance, 3DGS serves as a persistent spatial memory that endows the agent with textitSpatial Recollection: the ability to render photorealistic novel views from optimal, previously unoccupied viewpoints. To operationalize this, GSMem employs a retrieval mechanism that simultaneously leverages parallel object-level scene graphs and semantic-level language fields. This complementary design robustly localizes target regions, enabling the agent to ``hallucinate'' optimal views for high-fidelity Vision-Language Model (VLM) reasoning. Furthermore, we introduce a hybrid exploration strategy that combines VLM-driven semantic scoring with a 3DGS-based coverage objective, balancing task-aware exploration with geometric coverage. Extensive experiments on embodied question answering and lifelong navigation demonstrate the robustness and effectiveness of our framework
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 34 references that are in the map. First posted on arXiv 19 Mar 2026.
Builds on
29 of 34 references.
Closest ancestors · 1
Same direction, newest first
Other references · 28
- 2023GPT-4 Technical Reportcited by 18,132
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2025Qwen3 Technical Reportcited by 8,195
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2024GPT-4o System Cardcited by 4,467
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
- 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
- 2023LangSplat: 3D Language Gaussian Splattingcited by 410
- 2023Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fieldscited by 361
- 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
16 more
- 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 201
- 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
- 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Modelscited by 160
- 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 106
- 2023SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentationcited by 95
- 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
- 2023GOAT: GO to Any Thingcited by 89
- 2024Explore until Confident: Efficient Exploration for Embodied Question Answeringcited by 74
- 20243D-Mem: 3D Scene Memory for Embodied Exploration and Reasoningcited by 73
- 2023FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Informationcited by 63
- 2023Foundations of Spatial Perception for Robotics: Hierarchical Representations and Real-time Systemscited by 60
- 2025Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigationcited by 50
- 2024TANGO: Training-free Embodied AI Agents for Open-world Taskscited by 26
- 2024Occam's LGS: An Efficient Approach for Language Gaussian Splattingcited by 15
- 2025Online Language Splattingcited by 10
- 2025DyNaVLM: Zero-Shot Vision-Language Navigation System with Dynamic Viewpoints and Self-Refining Graph Memorycited by 9
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026What Spatial Memory Must Store: Occlusion as the Test for Language-Agent Memorycited by 0
- 2026Active Spatial Inspection for Effective and Efficient Embodied Explorationcited by 0
- 2026CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping with an LLM Agent for Immersive Collaborative VR Teleoperationcited by 0
- 2026Never Look Back: Understanding Persistence in 3D Object Memory from Egocentric Videoscited by 0
- 2026Rendering-Free Lookahead for Question-Guided Active Visioncited by 0
- 2026Rendering-Free Lookahead for Question-Guided Active Vision8 Oct 2026
- 2026Never Look Back: Understanding Persistence in 3D Object Memory from Egocentric Videos7 Oct 2026
- 2026CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping with an LLM Agent for Immersive Collaborative VR Teleoperation25 Sep 2026
- 2026Active Spatial Inspection for Effective and Efficient Embodied Exploration18 Sep 2026
- 2026What Spatial Memory Must Store: Occlusion as the Test for Language-Agent Memory9 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026RenderMem: Rendering as Spatial Memory Retrieval
- 20263DGSNav: Enhancing Vision-Language Model Reasoning for Object Navigation via Active 3D Gaussian Splatting
- 2026SplatReasoner: Enhancing Embodied Reasoning and Grounding by Novel View Synthesis
- 2025Vision to Geometry: 3D Spatial Memory for Sequential Embodied MLLM Reasoning and Exploration
- 2026E3VS-Bench: A Benchmark for Viewpoint-Dependent Active Perception in 3D Gaussian Splatting Scenes
- 2026RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics
- 2026EvoMemNav: Efficient Self-Evolving Fine-Grained Memory for Zero-Shot Embodied Navigation
- 2026SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments