# VI-Net: Boosting Category-level 6D Object Pose Estimation via Learning Decoupled Rotations on the Spherical Representations

arXiv: https://arxiv.org/abs/2308.09916 · HTML: https://mlsanity.com/paper/2308.09916/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 40 references that are in the map.
- First posted on arXiv 19 Aug 2023.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion](https://mlsanity.com/paper/1901.04780/) (2019)
- [Learning SO(3) Equivariant Representations with Spherical CNNs](https://mlsanity.com/paper/1711.06721/) (2017)
- [6-DOF GraspNet: Variational Grasp Generation for Object Manipulation](https://mlsanity.com/paper/1905.10520/) (2019)
- [Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation](https://mlsanity.com/paper/2007.08454/) (2020)
- [FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation](https://mlsanity.com/paper/2103.02242/) (2021)
- [Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection](https://mlsanity.com/paper/1903.01864/) (2019)
- [FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanism](https://mlsanity.com/paper/2103.07054/) (2021)
- [DualPoseNet: Category-level 6D Object Pose and Size Estimation Using Dual Pose Network with Refined Learning of Pose Consistency](https://mlsanity.com/paper/2103.06526/) (2021)
- [GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting](https://mlsanity.com/paper/2203.07918/) (2022)
- [Category-Level 6D Object Pose and Size Estimation using Self-Supervised Deep Prior Deformation Networks](https://mlsanity.com/paper/2207.05444/) (2022)
- [CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimation](https://mlsanity.com/paper/2203.01929/) (2022)
- [Category-Level 6D Object Pose Estimation via Cascaded Relation and Recurrent Reconstruction Networks](https://mlsanity.com/paper/2108.08755/) (2021)
- [SAR-Net: Shape Alignment and Recovery Network for Category-level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/2106.14193/) (2021)
- [VISTA: Boosting 3D Object Detection via Dual Cross-VIew SpaTial Attention](https://mlsanity.com/paper/2203.09704/) (2022)
- [OVE6D: Object Viewpoint Encoding for Depth-based 6D Object Pose Estimation](https://mlsanity.com/paper/2203.01072/) (2022)
- [RBP-Pose: Residual Bounding Box Projection for Category-Level Pose Estimation](https://mlsanity.com/paper/2208.00237/) (2022)
- [SSP-Pose: Symmetry-Aware Shape Prior Deformation for Direct Category-Level Object Pose Estimation](https://mlsanity.com/paper/2208.06661/) (2022)
- [CATRE: Iterative Point Clouds Alignment for Category-level Object Pose Refinement](https://mlsanity.com/paper/2207.08082/) (2022)
- [Grasp Proposal Networks: An End-to-End Solution for Visual Learning of Robotic Grasps](https://mlsanity.com/paper/2009.12606/) (2020)
- [ACR-Pose: Adversarial Canonical Representation Reconstruction Network for Category Level 6D Object Pose Estimation](https://mlsanity.com/paper/2111.10524/) (2021)
- [Sparse Steerable Convolutions: An Efficient Learning of SE(3)-Equivariant Features for Estimation and Tracking of Object Poses in 3D Space](https://mlsanity.com/paper/2111.07383/) (2021)

## Built on by (most cited first, 30 of 41)

- [SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation](https://mlsanity.com/paper/2311.15707/) (2023)
- [SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimation](https://mlsanity.com/paper/2311.11125/) (2023)
- [Deep Learning-Based Object Pose Estimation: A Comprehensive Survey](https://mlsanity.com/paper/2405.07801/) (2024)
- [GCE-Pose: Global Context Enhancement for Category-level Object Pose Estimation](https://mlsanity.com/paper/2502.04293/) (2025)
- [Diff9D: Diffusion-Based Domain-Generalized Category-Level 9-DoF Object Pose Estimation](https://mlsanity.com/paper/2502.02525/) (2025)
- [LaPose: Laplacian Mixture Shape Modeling for RGB-Based Category-Level Object Pose Estimation](https://mlsanity.com/paper/2409.15727/) (2024)
- [Zero123-6D: Zero-shot Novel View Synthesis for RGB Category-level 6D Pose Estimation](https://mlsanity.com/paper/2403.14279/) (2024)
- [CleanPose: Category-Level Object Pose Estimation via Causal Learning and Knowledge Distillation](https://mlsanity.com/paper/2502.01312/) (2025)
- [Learning Shape-Independent Transformation via Spherical Representations for Category-Level Object Pose Estimation](https://mlsanity.com/paper/2503.13926/) (2025)
- [GIVEPose: Gradual Intra-class Variation Elimination for RGB-based Category-Level Object Pose Estimation](https://mlsanity.com/paper/2503.15110/) (2025)
- [PicoPose: Progressive Pixel-to-Pixel Correspondence Learning for Novel Object Pose Estimation](https://mlsanity.com/paper/2504.02617/) (2025)
- [MonoDiff9D: Monocular Category-Level 9D Object Pose Estimation via Diffusion Model](https://mlsanity.com/paper/2504.10433/) (2025)
- [ComPose: A Unified Completion-Pose Framework for Robust Category-Level Object Pose Estimation](https://mlsanity.com/paper/2605.25553/) (2026)
- [Towards Real-World Aerial Vision Guidance with Categorical 6D Pose Tracker](https://mlsanity.com/paper/2401.04377/) (2024)
- [Advancing 6-DoF Instrument Pose Estimation in Variable X-Ray Imaging Geometries](https://mlsanity.com/paper/2405.11677/) (2024)
- [Structure-Aware Correspondence Learning for Relative Pose Estimation](https://mlsanity.com/paper/2503.18671/) (2025)
- [Diffusion-Driven Self-Supervised Learning for Shape Reconstruction and Pose Estimation](https://mlsanity.com/paper/2403.12728/) (2024)
- [fCOP: Focal Length Estimation from Category-level Object Priors](https://mlsanity.com/paper/2409.19641/) (2024)
- [Object Pose Estimation Using Implicit Representation For Transparent Objects](https://mlsanity.com/paper/2410.13465/) (2024)
- [Instance-Adaptive Keypoint Learning with Local-to-Global Geometric Aggregation for Category-Level Object Pose Estimation](https://mlsanity.com/paper/2504.15134/) (2025)
- [Joint Learning of Pose Regression and Denoising Diffusion with Score Scaling Sampling for Category-level 6D Pose Estimation](https://mlsanity.com/paper/2510.04125/) (2025)
- [OPFormer: Object Pose Estimation leveraging foundation model with geometric encoding](https://mlsanity.com/paper/2511.12614/) (2025)
- [CanoVerse: 3D Object Scalable Canonicalization and Dataset for Generation and Pose](https://mlsanity.com/paper/2603.07144/) (2026)
- [Flow6D: Discrete-to-Continuous Flow Matching for Efficient and Accurate Category-Level 6D Pose Estimation](https://mlsanity.com/paper/2606.23293/) (2026)
- [MemPose: Category-level Object Pose Estimation with Memory](https://mlsanity.com/paper/2607.04930/) (2026)
- [Towards Human-Level 3D Relative Pose Estimation: Generalizable, Training-Free, with Single Reference](https://mlsanity.com/paper/2406.18453/) (2024)
- [Category Level 6D Object Pose Estimation from a Single RGB Image using Diffusion](https://mlsanity.com/paper/2412.11420/) (2024)
- [Universal Features Guided Zero-Shot Category-Level Object Pose Estimation](https://mlsanity.com/paper/2501.02831/) (2025)
- [DynOPETs: A Versatile Benchmark for Dynamic Object Pose Estimation and Tracking in Moving Camera Scenarios](https://mlsanity.com/paper/2503.19625/) (2025)
- [RCGNet: RGB-based Category-Level 6D Object Pose Estimation with Geometric Guidance](https://mlsanity.com/paper/2508.13623/) (2025)

## Abstract (arXiv)

Rotation estimation of high precision from an RGB-D object observation is a huge challenge in 6D object pose estimation, due to the difficulty of learning in the non-linear space of SO(3). In this paper, we propose a novel rotation estimation network, termed as VI-Net, to make the task easier by decoupling the rotation as the combination of a viewpoint rotation and an in-plane rotation. More specifically, VI-Net bases the feature learning on the sphere with two individual branches for the estimates of two factorized rotations, where a V-Branch is employed to learn the viewpoint rotation via binary classification on the spherical signals, while another I-Branch is used to estimate the in-plane rotation by transforming the signals to view from the zenith direction. To process the spherical signals, a Spherical Feature Pyramid Network is constructed based on a novel design of SPAtial Spherical Convolution (SPA-SConv), which settles the boundary problem of spherical signals via feature padding and realizesviewpoint-equivariant feature extraction by symmetric convolutional operations. We apply the proposed VI-Net to the challenging task of category-level 6D object pose estimation for predicting the poses of unknown objects without available CAD models; experiments on the benchmarking datasets confirm the efficacy of our method, which outperforms the existing ones with a large margin in the regime of high precision.