# 3DGS-DET: Empower 3D Gaussian Splatting with Boundary Guidance and Box-Focused Sampling for Indoor 3D Object Detection

arXiv: https://arxiv.org/abs/2410.01647 · HTML: https://mlsanity.com/paper/2410.01647/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 77 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Builds on (most cited first, 30 of 62)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2403.14627/) (2024)
- [ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data](https://mlsanity.com/paper/2111.08897/) (2021)
- [LangSplat: 3D Language Gaussian Splatting](https://mlsanity.com/paper/2312.16084/) (2023)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields](https://mlsanity.com/paper/2312.03203/) (2023)
- [PIXOR: Real-time 3D Object Detection from Point Clouds](https://mlsanity.com/paper/1902.06326/) (2019)
- [Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection](https://mlsanity.com/paper/2303.11926/) (2023)
- [Voxel Transformer for 3D Object Detection](https://mlsanity.com/paper/2109.02497/) (2021)
- [Differentiable Surface Splatting for Point-based Geometry Processing](https://mlsanity.com/paper/1906.04173/) (2019)
- [VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking](https://mlsanity.com/paper/2303.11301/) (2023)
- [3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans](https://mlsanity.com/paper/1812.07003/) (2018)
- [VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction](https://mlsanity.com/paper/2402.17427/) (2024)
- [Language Embedded 3D Gaussians for Open-Vocabulary Scene Understanding](https://mlsanity.com/paper/2311.18482/) (2023)
- [CityGaussian: Real-time High-quality Large-Scale Scene Rendering with Gaussians](https://mlsanity.com/paper/2404.01133/) (2024)
- [SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation](https://mlsanity.com/paper/1711.08588/) (2017)

## Built on by (most cited first, 13 of 13)

- [Neural Fields in Robotics: A Survey](https://mlsanity.com/paper/2410.20220/) (2024)
- [A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generation](https://mlsanity.com/paper/2508.09977/) (2025)
- [ImOV3D: Learning Open-Vocabulary Point Clouds 3D Object Detection from Only 2D Images](https://mlsanity.com/paper/2410.24001/) (2024)
- [RaGS: Unleashing 3D Gaussian Splatting from 4D Radar and Monocular Cues for 3D Object Detection](https://mlsanity.com/paper/2507.19856/) (2025)
- [MomaGraph: State-Aware Unified Scene Graphs with Vision-Language Model for Embodied Task Planning](https://mlsanity.com/paper/2512.16909/) (2025)
- [Progressive Gaussian Transformer with Anisotropy-aware Sampling for Open Vocabulary Occupancy Prediction](https://mlsanity.com/paper/2510.04759/) (2025)
- [GAP-MLLM: Geometry-Aligned Pre-training for Activating 3D Spatial Perception in Multimodal Large Language Models](https://mlsanity.com/paper/2603.16461/) (2026)
- [Lemon: A Unified and Scalable 3D Multimodal Model for Universal Spatial Understanding](https://mlsanity.com/paper/2512.12822/) (2025)
- [3D Skew Gaussian Splatting with Any Camera Trajectory Visualization Engine](https://mlsanity.com/paper/2605.18334/) (2026)
- [GaussianGAN: Real-Time Photorealistic controllable Human Avatars](https://mlsanity.com/paper/2509.01681/) (2025)
- [GVSynergy-Det: Synergistic Gaussian-Voxel Representations for Multi-View 3D Object Detection](https://mlsanity.com/paper/2512.23176/) (2025)
- [Open-Vocabulary BEV Segmentation with 3D-Aware Geometric Constraints](https://mlsanity.com/paper/2606.24353/) (2026)
- [CoCam4D: Geometry-Aware Cooperative 4D Perception for Camera-Only Autonomous Driving](https://mlsanity.com/paper/2610.11577/) (2026)

## Abstract (arXiv)

Neural Radiance Fields (NeRF) have been adapted for indoor 3D Object Detection (3DOD), offering a promising approach to indoor 3DOD via view-synthesis representation. But its implicit nature limits representational capacity. Recently, 3D Gaussian Splatting (3DGS) has emerged as an explicit 3D representation that addresses the limitation. This work introduces 3DGS into indoor 3DOD for the first time, identifying two main challenges: (i) Ambiguous spatial distribution of Gaussian blobs -- 3DGS primarily relies on 2D pixel-level supervision, resulting in unclear 3D spatial distribution of Gaussian blobs and poor differentiation between objects and background, which hinders indoor 3DOD; (ii) Excessive background blobs -- 2D images typically include numerous background pixels, leading to densely reconstructed 3DGS with many noisy Gaussian blobs representing the background, negatively affecting detection. To tackle (i), we leverage the fact that 3DGS reconstruction is derived from 2D images, and propose an elegant solution by incorporating 2D Boundary Guidance to significantly enhance the spatial distribution of Gaussian blobs, resulting in clearer differentiation between objects and their background (please see fig:teaser). To address (ii), we propose a Box-Focused Sampling strategy using 2D boxes to generate object probability distribution in 3D space, allowing effective probabilistic sampling in 3D to retain more object blobs and reduce noisy background blobs. Benefiting from these innovations, 3DGS-DET significantly outperforms the state-of-the-art NeRF-based method, NeRF-Det++, achieving improvements of +6.0 on mAP@0.25 and +7.8 on mAP@0.5 for the ScanNet, and the +14.9 on mAP@0.25 for the ARKITScenes.