# ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splatting

arXiv: https://arxiv.org/abs/2507.15454 · HTML: https://mlsanity.com/paper/2507.15454/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 59 references that are in the map.
- First posted on arXiv 21 Jul 2025.

## Builds on (most cited first, 30 of 51)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [2D Gaussian Splatting for Geometrically Accurate Radiance Fields](https://mlsanity.com/paper/2403.17888/) (2024)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes](https://mlsanity.com/paper/2308.11417/) (2023)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP](https://mlsanity.com/paper/2210.04150/) (2022)
- [LERF: Language Embedded Radiance Fields](https://mlsanity.com/paper/2303.09553/) (2023)
- [LangSplat: 3D Language Gaussian Splatting](https://mlsanity.com/paper/2312.16084/) (2023)
- [Gaussian Grouping: Segment and Edit Anything in 3D Scenes](https://mlsanity.com/paper/2312.00732/) (2023)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [OpenMask3D: Open-Vocabulary 3D Instance Segmentation](https://mlsanity.com/paper/2306.13631/) (2023)
- [gsplat: An Open-Source Library for Gaussian Splatting](https://mlsanity.com/paper/2409.06765/) (2024)
- [AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Views](https://mlsanity.com/paper/2505.23716/) (2025)
- [Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians](https://mlsanity.com/paper/2403.17898/) (2024)
- [Tracking Anything with Decoupled Video Segmentation](https://mlsanity.com/paper/2309.03903/) (2023)
- [Segment Any 3D Gaussians](https://mlsanity.com/paper/2312.00860/) (2023)
- [SAM3D: Segment Anything in 3D Scenes](https://mlsanity.com/paper/2306.03908/) (2023)
- [OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary Understanding](https://mlsanity.com/paper/2406.02058/) (2024)

## Built on by (most cited first, 25 of 25)

- [A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generation](https://mlsanity.com/paper/2508.09977/) (2025)
- [MeshSplat: Generalizable Sparse-View Surface Reconstruction via Gaussian Splatting](https://mlsanity.com/paper/2508.17811/) (2025)
- [Consistent Instance Field for Dynamic Scene Understanding](https://mlsanity.com/paper/2512.14126/) (2025)
- [OpenVoxel: Training-Free Grouping and Captioning Voxels for Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2601.09575/) (2026)
- [B3-Seg: Camera-Free, Training-Free 3DGS Segmentation via Analytic EIG and Beta-Bernoulli Bayesian Updates](https://mlsanity.com/paper/2602.17134/) (2026)
- [EAG-PT: Emission-Aware Gaussians and Path Tracing for Diffuse Indoor Scene Reconstruction and Editing](https://mlsanity.com/paper/2601.23065/) (2026)
- [LangGS-SLAM: Real-Time Language-Feature Gaussian Splatting SLAM](https://mlsanity.com/paper/2602.06991/) (2026)
- [Informative Object-centric Next Best View for Object-aware 3D Gaussian Splatting in Cluttered Scenes](https://mlsanity.com/paper/2602.08266/) (2026)
- [Scene-Agnostic Object-Centric Representation Learning for 3D Gaussian Splatting](https://mlsanity.com/paper/2604.09045/) (2026)
- [Learning Action-Conditional and Object-Centric Gaussian Splatting World Models for Rigid Objects](https://mlsanity.com/paper/2606.01950/) (2026)
- [GASE: Gaussian Splatting-Based Automated System for Reconstructing Embodied-Simulation Environments](https://mlsanity.com/paper/2606.17520/) (2026)
- [Scenes as Objects, Not Primitives: Instance-Structured 3D Tokenization from Unposed Views](https://mlsanity.com/paper/2606.29513/) (2026)
- [Relation-Centric Open-Vocabulary 3D Gaussian Segmentation](https://mlsanity.com/paper/2607.01140/) (2026)
- [First Shape, Then Meaning: Efficient Geometry and Semantics Learning for Indoor Reconstruction](https://mlsanity.com/paper/2605.03463/) (2026)
- [Kinematics-Driven Gaussian Shape Deformation for Blurry Monocular Dynamic Scenes](https://mlsanity.com/paper/2605.08635/) (2026)
- [OP2GS: Object-Aware 3D Gaussian Splatting with Dual-Opacity Primitives](https://mlsanity.com/paper/2605.20044/) (2026)
- [DGSG-Mind: Dynamic 3D Gaussian Scene Graphs for Long-Term Scene Understanding and Grounding](https://mlsanity.com/paper/2605.29879/) (2026)
- [SemDynReg: Semantics-Guided Deformation Regularization for Dynamic 3D Gaussian Splatting](https://mlsanity.com/paper/2606.28656/) (2026)
- [Embodied Operators and Benchmarking: Toward Reusable and Deployable Embodied Intelligence Systems](https://mlsanity.com/paper/2607.03283/) (2026)
- [MonoVoc: Decoupling Geometry and Semantics for Lightweight Monocular Open-Vocabulary 3D Gaussians](https://mlsanity.com/paper/2607.28300/) (2026)
- [Seed2GS: Camera-Free, Training-Free Object Extraction from 3D Gaussian Scenes via a Single Reference-View Grounding](https://mlsanity.com/paper/2608.11928/) (2026)
- [Hybrid Gaussians for Robust Open-Vocabulary 3D Segmentation with Multi-View Object Association and Boundary Refinement](https://mlsanity.com/paper/2609.27462/) (2026)
- [PePESeg3D: Perception Prior Enhances Multi-Scale Segmentation for 3D Gaussian Splatting](https://mlsanity.com/paper/2609.28645/) (2026)
- [RelationVGGT: Visual Geometry Transformers for 3D Spatial Relation Segmentation](https://mlsanity.com/paper/2610.00970/) (2026)
- [DIGS: Distribution-Informed Gaussian Splatting for Training-Free Open-Vocabulary 3D Segmentation](https://mlsanity.com/paper/2610.16177/) (2026)

## Abstract (arXiv)

3D Gaussian Splatting is renowned for its high-fidelity reconstructions and real-time novel view synthesis, yet its lack of semantic understanding limits object-level perception. In this work, we propose ObjectGS, an object-aware framework that unifies 3D scene reconstruction with semantic understanding. Instead of treating the scene as a unified whole, ObjectGS models individual objects as local anchors that generate neural Gaussians and share object IDs, enabling precise object-level reconstruction. During training, we dynamically grow or prune these anchors and optimize their features, while a one-hot ID encoding with a classification loss enforces clear semantic constraints. We show through extensive experiments that ObjectGS not only outperforms state-of-the-art methods on open-vocabulary and panoptic segmentation tasks, but also integrates seamlessly with applications like mesh extraction and scene editing. Project page: https://ruijiezhu94.github.io/ObjectGS_page