# GFreeDet: Exploiting Gaussian Splatting and Foundation Models for Model-free Unseen Object Detection in the BOP Challenge 2024

arXiv: https://arxiv.org/abs/2412.01552 · HTML: https://mlsanity.com/paper/2412.01552/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 20 references that are in the map.
- First posted on arXiv 2 Dec 2024.

## Builds on (most cited first, 16 of 16)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Fast Segment Anything](https://mlsanity.com/paper/2306.12156/) (2023)
- [gsplat: An Open-Source Library for Gaussian Splatting](https://mlsanity.com/paper/2409.06765/) (2024)
- [HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos](https://mlsanity.com/paper/2411.19167/) (2024)
- [SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation](https://mlsanity.com/paper/2311.15707/) (2023)
- [CNOS: A Strong Baseline for CAD-based Novel Object Segmentation](https://mlsanity.com/paper/2307.11067/) (2023)
- [6-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmark](https://mlsanity.com/paper/2203.05701/) (2022)
- [BOP Challenge 2023 on Detection, Segmentation and Pose Estimation of Seen and Unseen Rigid Objects](https://mlsanity.com/paper/2403.09799/) (2024)
- [GaussianObject: High-Quality 3D Object Reconstruction from Four Views with Gaussian Splatting](https://mlsanity.com/paper/2402.10259/) (2024)
- [HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructions](https://mlsanity.com/paper/2308.01477/) (2023)
- [Introducing HOT3D: An Egocentric Dataset for 3D Hand and Object Tracking](https://mlsanity.com/paper/2406.09598/) (2024)
- [Fisheye-GS: Lightweight and Extensible Gaussian Splatting Module for Fisheye Cameras](https://mlsanity.com/paper/2409.04751/) (2024)

## Built on by (most cited first, 3 of 3)

- [BOP Challenge 2024 on Model-Based and Model-Free 6D Object Pose Estimation](https://mlsanity.com/paper/2504.02812/) (2025)
- [Street Gaussians without 3D Object Tracker](https://mlsanity.com/paper/2412.05548/) (2024)
- [Finding NeMO: A Geometry-Aware Representation of Template Views for Few-Shot Perception](https://mlsanity.com/paper/2602.04343/) (2026)

## Abstract (arXiv)

We present GFreeDet, an unseen object detection approach that leverages Gaussian splatting and vision Foundation models under model-free setting. Unlike existing methods that rely on predefined CAD templates, GFreeDet reconstructs objects directly from reference videos using Gaussian splatting, enabling robust detection of novel objects without prior 3D models. Evaluated on the BOP-H3 benchmark, GFreeDet achieves comparable performance to CAD-based methods, demonstrating the viability of model-free detection for mixed reality (MR) applications. Notably, GFreeDet won the best overall method and the best fast method awards in the model-free 2D detection track at BOP Challenge 2024.