# GS-Pose: Generalizable Segmentation-based 6D Object Pose Estimation with 3D Gaussian Splatting

arXiv: https://arxiv.org/abs/2403.10683 · HTML: https://mlsanity.com/paper/2403.10683/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 60 references that are in the map.
- First posted on arXiv 15 Mar 2024.

## Builds on (most cited first, 30 of 45)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects](https://mlsanity.com/paper/2312.08344/) (2023)
- [PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1812.11788/) (2018)
- [CosyPose: Consistent multi-view multi-object 6D pose estimation](https://mlsanity.com/paper/2008.08465/) (2020)
- [DeepIM: Deep Iterative Matching for 6D Pose Estimation](https://mlsanity.com/paper/1804.00175/) (2018)
- [MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare](https://mlsanity.com/paper/2212.06870/) (2022)
- [Implicit 3D Orientation Learning for 6D Object Detection from RGB Images](https://mlsanity.com/paper/1902.01275/) (2019)
- [OnePose: One-Shot Object Pose Estimation without CAD Models](https://mlsanity.com/paper/2205.12257/) (2022)
- [EPOS: Estimating 6D Pose of Objects with Symmetries](https://mlsanity.com/paper/2004.00605/) (2020)
- [Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/2001.09322/) (2020)
- [FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanism](https://mlsanity.com/paper/2103.07054/) (2021)
- [Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images](https://mlsanity.com/paper/2204.10776/) (2022)
- [SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation](https://mlsanity.com/paper/2311.15707/) (2023)
- [ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimation](https://mlsanity.com/paper/2203.09418/) (2022)
- [OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Models](https://mlsanity.com/paper/2301.07673/) (2023)
- [GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondence](https://mlsanity.com/paper/2311.14155/) (2023)
- [FoundPose: Unseen Object Pose Estimation with Foundation Features](https://mlsanity.com/paper/2311.18809/) (2023)
- [Self-supervised 6D Object Pose Estimation for Robot Manipulation](https://mlsanity.com/paper/1909.10159/) (2019)
- [Category Level Object Pose Estimation via Neural Analysis-by-Synthesis](https://mlsanity.com/paper/2008.08145/) (2020)
- [SurfEmb: Dense and Continuous Correspondence Distributions for Object Pose Estimation with Learnt Surface Embeddings](https://mlsanity.com/paper/2111.13489/) (2021)
- [LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimation](https://mlsanity.com/paper/1912.00416/) (2019)

## Built on by (most cited first, 15 of 15)

- [3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities](https://mlsanity.com/paper/2407.17418/) (2024)
- [Reconstructing In-the-Wild Open-Vocabulary Human-Object Interactions](https://mlsanity.com/paper/2503.15898/) (2025)
- [ConceptPose: Training-Free Zero-Shot Object Pose Estimation using Concept Vectors](https://mlsanity.com/paper/2512.09056/) (2025)
- [Object Gaussian for Monocular 6D Pose Estimation from Sparse Views](https://mlsanity.com/paper/2409.02581/) (2024)
- [GS2Pose: Two-stage 6D Object Pose Estimation Guided by Gaussian Splatting](https://mlsanity.com/paper/2411.03807/) (2024)
- [BoxDreamer: Dreaming Box Corners for Generalizable Object Pose Estimation](https://mlsanity.com/paper/2504.07955/) (2025)
- [T-Rex: Task-Adaptive Spatial Representation Extraction for Robotic Manipulation with Vision-Language Models](https://mlsanity.com/paper/2506.19498/) (2025)
- [GSGTrack: Gaussian Splatting-Guided Object Pose Tracking from RGB Videos](https://mlsanity.com/paper/2412.02267/) (2024)
- [GMatch: A Lightweight, Geometry-Constrained Keypoint Matcher for Zero-Shot 6DoF Pose Estimation in Robotic Grasp Tasks](https://mlsanity.com/paper/2505.16144/) (2025)
- [Eff-GRot: Efficient and Generalizable Rotation Estimation with Transformers](https://mlsanity.com/paper/2512.18784/) (2025)
- [Exploring 6D Object Pose Estimation with Deformation](https://mlsanity.com/paper/2604.06720/) (2026)
- [MAPRPose: Mask-Aware Proposal and Amodal Refinement for Multi-Object 6D Pose Estimation](https://mlsanity.com/paper/2604.20650/) (2026)
- [Pose Anything Anywhere:Model-free Object Poses from Arbitrary References](https://mlsanity.com/paper/2606.23634/) (2026)
- [LEGAU: Learning Semantic Gaussian Priors for Scalable Category-level Pose Estimation](https://mlsanity.com/paper/2609.35046/) (2026)
- [NovaSplat: Model-Free 6D Object Pose Estimation via 3D Gaussian Splatting Onboarding and Refinement](https://mlsanity.com/paper/2610.13309/) (2026)

## Abstract (arXiv)

This paper introduces GS-Pose, a unified framework for localizing and estimating the 6D pose of novel objects. GS-Pose begins with a set of posed RGB images of a previously unseen object and builds three distinct representations stored in a database. At inference, GS-Pose operates sequentially by locating the object in the input image, estimating its initial 6D pose using a retrieval approach, and refining the pose with a render-and-compare method. The key insight is the application of the appropriate object representation at each stage of the process. In particular, for the refinement step, we leverage 3D Gaussian splatting, a novel differentiable rendering technique that offers high rendering speed and relatively low optimization time. Off-the-shelf toolchains and commodity hardware, such as mobile phones, can be used to capture new objects to be added to the database. Extensive evaluations on the LINEMOD and OnePose-LowTexture datasets demonstrate excellent performance, establishing the new state-of-the-art. Project page: https://dingdingcai.github.io/gs-pose.