# Accurate and efficient zero-shot 6D pose estimation with frozen foundation models

arXiv: https://arxiv.org/abs/2506.09784 · HTML: https://mlsanity.com/paper/2506.09784/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 68 of its 84 references that are in the map.
- First posted on arXiv 11 Jun 2025.

## Builds on (most cited first, 30 of 68)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [Point Transformer V3: Simpler, Faster, Stronger](https://mlsanity.com/paper/2312.10035/) (2023)
- [Geometric deep learning on graphs and manifolds using mixture model CNNs](https://mlsanity.com/paper/1611.08402/) (2016)
- [PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies](https://mlsanity.com/paper/2206.04670/) (2022)
- [FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects](https://mlsanity.com/paper/2312.08344/) (2023)
- [Point Transformer V2: Grouped Vector Attention and Partition-based Pooling](https://mlsanity.com/paper/2210.05666/) (2022)
- [PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1812.11788/) (2018)
- [Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework](https://mlsanity.com/paper/2202.07123/) (2022)
- [DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion](https://mlsanity.com/paper/1901.04780/) (2019)
- [Fast Segment Anything](https://mlsanity.com/paper/2306.12156/) (2023)
- [BOP: Benchmark for 6D Object Pose Estimation](https://mlsanity.com/paper/1808.08319/) (2018)
- [T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objects](https://mlsanity.com/paper/1701.05498/) (2017)
- [Geometric Transformer for Fast and Robust Point Cloud Registration](https://mlsanity.com/paper/2202.06688/) (2022)
- [Patches Are All You Need?](https://mlsanity.com/paper/2201.09792/) (2022)
- [MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare](https://mlsanity.com/paper/2212.06870/) (2022)

## Built on by (most cited first, 13 of 13)

- [WALDO: Where Unseen Model-based 6D Pose Estimation Meets Occlusion](https://mlsanity.com/paper/2511.15874/) (2025)
- [Color-Pair Guided Robust Zero-Shot 6D Pose Estimation and Tracking of Cluttered Objects on Edge Devices](https://mlsanity.com/paper/2509.23647/) (2025)
- [Geo6DPose: Fast Zero-Shot 6D Object Pose Estimation via Geometry-Filtered Feature Matching](https://mlsanity.com/paper/2512.10674/) (2025)
- [AlignPose: Generalizable 6D Pose Estimation via Multi-view Feature-metric Alignment](https://mlsanity.com/paper/2512.20538/) (2025)
- [Pickalo: Leveraging 6D Pose Estimation for Low-Cost Industrial Bin Picking](https://mlsanity.com/paper/2604.04690/) (2026)
- [MAPRPose: Mask-Aware Proposal and Amodal Refinement for Multi-Object 6D Pose Estimation](https://mlsanity.com/paper/2604.20650/) (2026)
- [Semantic Prior Guided One-View 6D Pose Estimation for Novel Objects](https://mlsanity.com/paper/2605.07023/) (2026)
- [xperception -- Making Robotic Grasping Easier](https://mlsanity.com/paper/2607.16312/) (2026)
- [Foundational feature fusion for conditional flow matching in 6D pose estimation](https://mlsanity.com/paper/2608.29183/) (2026)
- [Prior-free relative 6D pose estimation of multiple object instances](https://mlsanity.com/paper/2609.08949/) (2026)
- [G6D: Geometric Learning-Free RGB-D 6D Pose Solver for Robotic Manipulation](https://mlsanity.com/paper/2609.23566/) (2026)
- [GRC-Pose: Generation-Reconstruction Correspondence for Prior-Free 6D Object Pose Tracking](https://mlsanity.com/paper/2609.39116/) (2026)
- [WAPR: A Foundation Model for Wide-Angle Refinement in Unseen Object Pose Estimation](https://mlsanity.com/paper/2610.09535/) (2026)

## Abstract (arXiv)

Estimating the 6D pose of objects from RGBD data is a fundamental problem in computer vision, with applications in robotics and augmented reality. A key challenge is achieving generalization to novel objects that were not seen during training. Most existing approaches address this by scaling up training on synthetic data tailored to the task, a process that demands substantial computational resources. But is task-specific training really necessary for accurate and efficient 6D pose estimation of novel objects? To answer No!, we introduce FreeZeV2, the second generation of FreeZe: a training-free method that achieves strong generalization to unseen objects by leveraging geometric and vision foundation models pre-trained on unrelated data. FreeZeV2 improves both accuracy and efficiency over FreeZe through three key contributions: (i) a sparse feature extraction strategy that reduces inference-time computation without sacrificing accuracy; (ii) a feature-aware scoring mechanism that improves both pose selection during RANSAC-based 3D registration and the final ranking of pose candidates; and (iii) a modular design that supports ensembles of instance segmentation models, increasing robustness to segmentation masks errors. We evaluate FreeZeV2 on the seven core datasets of the BOP Benchmark, where it establishes a new state-of-the-art in 6D pose estimation of unseen objects. When using the same segmentation masks, FreeZeV2 achieves a remarkable 8x speedup over FreeZe while also improving accuracy by 5%. When using ensembles of segmentation models, FreeZeV2 gains an additional 8% in accuracy while still running 2.5x faster than FreeZe. FreeZeV2 was awarded Best Overall Method at the BOP Challenge 2024.