# Image to Sphere: Learning Equivariant Features for Efficient Pose Prediction

arXiv: https://arxiv.org/abs/2302.13926 · HTML: https://mlsanity.com/paper/2302.13926/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 27 Feb 2023.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [Vector Neurons: A General Framework for SO(3)-Equivariant Networks](https://mlsanity.com/paper/2104.12229/) (2021)
- [Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects](https://mlsanity.com/paper/1809.10790/) (2018)
- [PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1911.04231/) (2019)
- [DPOD: 6D Pose Object Detector and Refiner](https://mlsanity.com/paper/1902.11020/) (2019)
- [Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views](https://mlsanity.com/paper/1505.05641/) (2015)
- [Equivariant Point Network for 3D Point Cloud Analysis](https://mlsanity.com/paper/2103.14147/) (2021)
- [Deep Regression on Manifolds: A 3D Rotation Case Study](https://mlsanity.com/paper/2103.16317/) (2021)
- [Explorations in Homeomorphic Variational Auto-Encoding](https://mlsanity.com/paper/1807.04689/) (2018)
- [Implicit-PDF: Non-Parametric Representation of Probability Distributions on the Rotation Manifold](https://mlsanity.com/paper/2106.05965/) (2021)
- [Viewpoints and Keypoints](https://mlsanity.com/paper/1411.6067/) (2014)
- [Learning Symmetric Embeddings for Equivariant World Models](https://mlsanity.com/paper/2204.11371/) (2022)
- [Deep Directional Statistics: Pose Estimation with Uncertainty Quantification](https://mlsanity.com/paper/1805.03430/) (2018)
- [Equivariant Multi-View Networks](https://mlsanity.com/paper/1904.00993/) (2019)
- [Deep Bingham Networks: Dealing with Uncertainty and Ambiguity in Pose Estimation](https://mlsanity.com/paper/2012.11002/) (2020)
- [Spherical Regression: Learning Viewpoints, Surface Normals and 3D Rotations on n-Spheres](https://mlsanity.com/paper/1904.05404/) (2019)
- [Probabilistic orientation estimation with matrix Fisher distributions](https://mlsanity.com/paper/2006.09740/) (2020)
- [CryoAI: Amortized Inference of Poses for Ab Initio Reconstruction of 3D Molecular Volumes from Real Cryo-EM Images](https://mlsanity.com/paper/2203.08138/) (2022)
- [A Mixed Classification-Regression Framework for 3D Pose Estimation from 2D Images](https://mlsanity.com/paper/1805.03225/) (2018)
- [Cross-Domain 3D Equivariant Image Embeddings](https://mlsanity.com/paper/1812.02716/) (2018)

## Built on by (most cited first, 18 of 18)

- [Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3D](https://mlsanity.com/paper/2401.12046/) (2024)
- [PUG: Photorealistic and Semantically Controllable Synthetic Data for Representation Learning](https://mlsanity.com/paper/2308.03977/) (2023)
- [A General Theory of Correct, Incorrect, and Extrinsic Equivariance](https://mlsanity.com/paper/2303.04745/) (2023)
- [Confronting Ambiguity in 6D Object Pose Estimation via Score-Based Diffusion on SE(3)](https://mlsanity.com/paper/2305.15873/) (2023)
- [Making Vision Transformers Truly Shift-Equivariant](https://mlsanity.com/paper/2305.16316/) (2023)
- [SpyroPose: SE(3) Pyramids for Object Pose Distribution Estimation](https://mlsanity.com/paper/2303.05308/) (2023)
- [SE(3)-Equivariant Diffusion Policy in Spherical Fourier Space](https://mlsanity.com/paper/2507.01723/) (2025)
- [Equivariant Single View Pose Prediction Via Induced and Restricted Representations](https://mlsanity.com/paper/2307.03704/) (2023)
- [3D Equivariant Visuomotor Policy Learning via Spherical Projection](https://mlsanity.com/paper/2505.16969/) (2025)
- [3D Equivariant Pose Regression via Direct Wigner-D Harmonics Prediction](https://mlsanity.com/paper/2411.00543/) (2024)
- [Equivariant Action Sampling for Reinforcement Learning and Planning](https://mlsanity.com/paper/2412.12237/) (2024)
- [BOP-Distrib: Revisiting 6D Pose Estimation Benchmarks for Better Evaluation under Visual Ambiguities](https://mlsanity.com/paper/2408.17297/) (2024)
- [Alignist: CAD-Informed Orientation Distribution Estimation by Fusing Shape and Correspondences](https://mlsanity.com/paper/2409.06683/) (2024)
- [Augmenting x-ray single particle imaging reconstruction with self-supervised machine learning](https://mlsanity.com/paper/2311.16652/) (2023)
- [Equivariant symmetry-aware head pose estimation for fetal MRI](https://mlsanity.com/paper/2512.04890/) (2025)
- [HACMatch Semi-Supervised Rotation Regression with Hardness-Aware Curriculum Pseudo Labeling](https://mlsanity.com/paper/2603.21583/) (2026)
- [ARCHER: Amortized cross-specimen pose estimation for cryo-electron microscopy](https://mlsanity.com/paper/2608.22029/) (2026)
- [Equivariant Visual-Tactile Diffusion Policy for Contact-Rich Manipulation](https://mlsanity.com/paper/2610.03333/) (2026)

## Abstract (arXiv)

Predicting the pose of objects from a single image is an important but difficult computer vision problem. Methods that predict a single point estimate do not predict the pose of objects with symmetries well and cannot represent uncertainty. Alternatively, some works predict a distribution over orientations in SO(3). However, training such models can be computation- and sample-inefficient. Instead, we propose a novel mapping of features from the image domain to the 3D rotation manifold. Our method then leverages SO(3) equivariant layers, which are more sample efficient, and outputs a distribution over rotations that can be sampled at arbitrary resolution. We demonstrate the effectiveness of our method at object orientation prediction, and achieve state-of-the-art performance on the popular PASCAL3D+ dataset. Moreover, we show that our method can model complex object symmetries, without any modifications to the parameters or loss function. Code is available at https://dmklee.github.io/image2sphere.