# Fusing Local Similarities for Retrieval-based 3D Orientation Estimation of Unseen Objects

arXiv: https://arxiv.org/abs/2203.08472 · HTML: https://mlsanity.com/paper/2203.08472/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 42 references that are in the map.
- First posted on arXiv 16 Mar 2022.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1812.11788/) (2018)
- [DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion](https://mlsanity.com/paper/1901.04780/) (2019)
- [DeepIM: Deep Iterative Matching for 6D Pose Estimation](https://mlsanity.com/paper/1804.00175/) (2018)
- [Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects](https://mlsanity.com/paper/1809.10790/) (2018)
- [Generalized Category Discovery](https://mlsanity.com/paper/2201.02609/) (2022)
- [T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objects](https://mlsanity.com/paper/1701.05498/) (2017)
- [PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation](https://mlsanity.com/paper/1711.10871/) (2017)
- [Implicit 3D Orientation Learning for 6D Object Detection from RGB Images](https://mlsanity.com/paper/1902.01275/) (2019)
- [Category-Level Articulated Object Pose Estimation](https://mlsanity.com/paper/1912.11913/) (2019)
- [kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation](https://mlsanity.com/paper/1903.06684/) (2019)
- [FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanism](https://mlsanity.com/paper/2103.07054/) (2021)
- [LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimation](https://mlsanity.com/paper/1912.00416/) (2019)
- [Single-Stage 6D Object Pose Estimation](https://mlsanity.com/paper/1911.08324/) (2019)
- [6-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypoints](https://mlsanity.com/paper/1910.10750/) (2019)
- [Learning Descriptors for Object Recognition and 3D Pose Estimation](https://mlsanity.com/paper/1502.05908/) (2015)
- [Multi-path Learning for Object Pose Estimation Across Domains](https://mlsanity.com/paper/1908.00151/) (2019)
- [Pose from Shape: Deep Pose Estimation for Arbitrary 3D Objects](https://mlsanity.com/paper/1906.05105/) (2019)
- [PoseContrast: Class-Agnostic Object Viewpoint Estimation in the Wild with Pose-Aware Contrastive Learning](https://mlsanity.com/paper/2105.05643/) (2021)

## Built on by (most cited first, 17 of 17)

- [Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images](https://mlsanity.com/paper/2204.10776/) (2022)
- [GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondence](https://mlsanity.com/paper/2311.14155/) (2023)
- [Deep Learning-Based Object Pose Estimation: A Comprehensive Survey](https://mlsanity.com/paper/2405.07801/) (2024)
- [NOPE: Novel Object Pose Estimation from a Single Image](https://mlsanity.com/paper/2303.13612/) (2023)
- [GS-Pose: Category-Level Object Pose Estimation via Geometric and Semantic Correspondence](https://mlsanity.com/paper/2311.13777/) (2023)
- [Object Pose Estimation via the Aggregation of Diffusion Features](https://mlsanity.com/paper/2403.18791/) (2024)
- [UNOPose: Unseen Object Pose Estimation with an Unposed RGB-D Reference Image](https://mlsanity.com/paper/2411.16106/) (2024)
- [3D-Aware Hypothesis & Verification for Generalizable Relative Object Pose Estimation](https://mlsanity.com/paper/2310.03534/) (2023)
- [OV9D: Open-Vocabulary Category-Level 9D Object Pose and Size Estimation](https://mlsanity.com/paper/2403.12396/) (2024)
- [6D Object Pose Tracking in Internet Videos for Robotic Manipulation](https://mlsanity.com/paper/2503.10307/) (2025)
- [LocPoseNet: Robust Location Prior for Unseen Object Pose Estimation](https://mlsanity.com/paper/2211.16290/) (2022)
- [Extreme Two-View Geometry From Object Poses with Diffusion Models](https://mlsanity.com/paper/2402.02800/) (2024)
- [Reasoning and Learning a Perceptual Metric for Self-Training of Reflective Objects in Bin-Picking with a Low-cost Camera](https://mlsanity.com/paper/2503.20207/) (2025)
- [Generalizable Single-view Object Pose Estimation by Two-side Generating and Matching](https://mlsanity.com/paper/2411.15860/) (2024)
- [DVMNet++: Rethinking Relative Pose Estimation for Unseen Objects](https://mlsanity.com/paper/2403.13683/) (2024)
- [Eff-GRot: Efficient and Generalizable Rotation Estimation with Transformers](https://mlsanity.com/paper/2512.18784/) (2025)
- [MixRI: Mixing Features of Reference Images for Novel Object Pose Estimation](https://mlsanity.com/paper/2601.06883/) (2026)

## Abstract (arXiv)

In this paper, we tackle the task of estimating the 3D orientation of previously-unseen objects from monocular images. This task contrasts with the one considered by most existing deep learning methods which typically assume that the testing objects have been observed during training. To handle the unseen objects, we follow a retrieval-based strategy and prevent the network from learning object-specific features by computing multi-scale local similarities between the query image and synthetically-generated reference images. We then introduce an adaptive fusion module that robustly aggregates the local similarities into a global similarity score of pairwise images. Furthermore, we speed up the retrieval process by developing a fast retrieval strategy. Our experiments on the LineMOD, LineMOD-Occluded, and T-LESS datasets show that our method yields a significantly better generalization to unseen objects than previous works. Our code and pre-trained models are available at https://sailor-z.github.io/projects/Unseen_Object_Pose.html.