# Spatial Feature Mapping for 6DoF Object Pose Estimation

arXiv: https://arxiv.org/abs/2206.01831 · HTML: https://mlsanity.com/paper/2206.01831/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 86 references that are in the map.
- First posted on arXiv 3 Jun 2022.

## Builds on (most cited first, 30 of 42)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1812.11788/) (2018)
- [Recent Advances in Convolutional Neural Networks](https://mlsanity.com/paper/1512.07108/) (2015)
- [DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion](https://mlsanity.com/paper/1901.04780/) (2019)
- [DeepIM: Deep Iterative Matching for 6D Pose Estimation](https://mlsanity.com/paper/1804.00175/) (2018)
- [3D Bounding Box Estimation Using Deep Learning and Geometry](https://mlsanity.com/paper/1612.00496/) (2016)
- [Learning SO(3) Equivariant Representations with Spherical CNNs](https://mlsanity.com/paper/1711.06721/) (2017)
- [Vision-Language Transformer and Query Generation for Referring Segmentation](https://mlsanity.com/paper/2108.05565/) (2021)
- [BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth](https://mlsanity.com/paper/1703.10896/) (2017)
- [Semantic Graph Convolutional Networks for 3D Human Pose Regression](https://mlsanity.com/paper/1904.03345/) (2019)
- [PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1911.04231/) (2019)
- [3D Hand Shape and Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/1903.00812/) (2019)
- [Implicit 3D Orientation Learning for 6D Object Detection from RGB Images](https://mlsanity.com/paper/1902.01275/) (2019)
- [MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis](https://mlsanity.com/paper/2103.14910/) (2021)
- [HOPE-Net: A Graph-based Model for Hand-Object Pose Estimation](https://mlsanity.com/paper/2004.00060/) (2020)
- [Boundary-Aware Feature Propagation for Scene Segmentation](https://mlsanity.com/paper/1909.00179/) (2019)
- [Segmentation-driven 6D Object Pose Estimation](https://mlsanity.com/paper/1812.02541/) (2018)
- [Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation](https://mlsanity.com/paper/1908.01491/) (2019)
- [Recovering the Unbiased Scene Graphs from the Biased Ones](https://mlsanity.com/paper/2107.02112/) (2021)
- [Explaining the Ambiguity of Object Detection and 6D Pose From Visual Data](https://mlsanity.com/paper/1812.00287/) (2018)
- [Semantic Correlation Promoted Shape-Variant Context for Segmentation](https://mlsanity.com/paper/1909.02651/) (2019)
- [Instance-Specific Feature Propagation for Referring Segmentation](https://mlsanity.com/paper/2204.12109/) (2022)
- [Flow-Guided Sparse Transformer for Video Deblurring](https://mlsanity.com/paper/2201.01893/) (2022)
- [Deep Learning of Local RGB-D Patches for 3D Object Detection and 6D Pose Estimation](https://mlsanity.com/paper/1607.06038/) (2016)

## Built on by (most cited first, 3 of 3)

- [Deep Learning-Based Object Pose Estimation: A Comprehensive Survey](https://mlsanity.com/paper/2405.07801/) (2024)
- [6D-Diff: A Keypoint Diffusion Framework for 6D Object Pose Estimation](https://mlsanity.com/paper/2401.00029/) (2023)
- [Challenges for Monocular 6D Object Pose Estimation in Robotics](https://mlsanity.com/paper/2307.12172/) (2023)

## Abstract (arXiv)

This work aims to estimate 6Dof (6D) object pose in background clutter. Considering the strong occlusion and background noise, we propose to utilize the spatial structure for better tackling this challenging task. Observing that the 3D mesh can be naturally abstracted by a graph, we build the graph using 3D points as vertices and mesh connections as edges. We construct the corresponding mapping from 2D image features to 3D points for filling the graph and fusion of the 2D and 3D features. Afterward, a Graph Convolutional Network (GCN) is applied to help the feature exchange among objects' points in 3D space. To address the problem of rotation symmetry ambiguity for objects, a spherical convolution is utilized and the spherical features are combined with the convolutional features that are mapped to the graph. Predefined 3D keypoints are voted and the 6DoF pose is obtained via the fitting optimization. Two scenarios of inference, one with the depth information and the other without it are discussed. Tested on the datasets of YCB-Video and LINEMOD, the experiments demonstrate the effectiveness of our proposed method.