# Equivariant Single View Pose Prediction Via Induced and Restricted Representations

arXiv: https://arxiv.org/abs/2307.03704 · HTML: https://mlsanity.com/paper/2307.03704/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 65 references that are in the map.
- First posted on arXiv 7 Jul 2023.

## Builds on (most cited first, 30 of 47)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [General E(2)-Equivariant Steerable CNNs](https://mlsanity.com/paper/1911.08251/) (2019)
- [3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data](https://mlsanity.com/paper/1807.02547/) (2018)
- [Steerable CNNs](https://mlsanity.com/paper/1612.08498/) (2016)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [Learning Steerable Filters for Rotation Equivariant CNNs](https://mlsanity.com/paper/1711.07289/) (2017)
- [A Survey of Structure from Motion](https://mlsanity.com/paper/1701.08493/) (2017)
- [Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs](https://mlsanity.com/paper/2302.03655/) (2023)
- [Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views](https://mlsanity.com/paper/1505.05641/) (2015)
- [Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs](https://mlsanity.com/paper/2003.05425/) (2020)
- [Object Scene Representation Transformer](https://mlsanity.com/paper/2206.06922/) (2022)
- [Monocular Depth Estimation using Diffusion Models](https://mlsanity.com/paper/2302.14816/) (2023)
- [Deep Regression on Manifolds: A 3D Rotation Case Study](https://mlsanity.com/paper/2103.16317/) (2021)
- [Explorations in Homeomorphic Variational Auto-Encoding](https://mlsanity.com/paper/1807.04689/) (2018)
- [Implicit-PDF: Non-Parametric Representation of Probability Distributions on the Rotation Manifold](https://mlsanity.com/paper/2106.05965/) (2021)
- [Viewpoints and Keypoints](https://mlsanity.com/paper/1411.6067/) (2014)
- [Reconstructing continuous distributions of 3D protein structure from cryo-EM images](https://mlsanity.com/paper/1909.05215/) (2019)
- [Learning Symmetric Embeddings for Equivariant World Models](https://mlsanity.com/paper/2204.11371/) (2022)
- [Invariant Slot Attention: Object Discovery with Slot-Centric Reference Frames](https://mlsanity.com/paper/2302.04973/) (2023)
- [Dense Depth Estimation in Monocular Endoscopy with Self-supervised Learning Methods](https://mlsanity.com/paper/1902.07766/) (2019)
- [A Wigner-Eckart Theorem for Group Equivariant Convolution Kernels](https://mlsanity.com/paper/2010.10952/) (2020)
- [Deep Directional Statistics: Pose Estimation with Uncertainty Quantification](https://mlsanity.com/paper/1805.03430/) (2018)
- [Equivariant Multi-View Networks](https://mlsanity.com/paper/1904.00993/) (2019)

## Built on by (most cited first, 6 of 6)

- [Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3D](https://mlsanity.com/paper/2401.12046/) (2024)
- [E(2)-Equivariant Graph Planning for Navigation](https://mlsanity.com/paper/2309.13043/) (2023)
- [3D Equivariant Pose Regression via Direct Wigner-D Harmonics Prediction](https://mlsanity.com/paper/2411.00543/) (2024)
- [Equivariant Action Sampling for Reinforcement Learning and Planning](https://mlsanity.com/paper/2412.12237/) (2024)
- [Equivariant symmetry-aware head pose estimation for fetal MRI](https://mlsanity.com/paper/2512.04890/) (2025)
- [HACMatch Semi-Supervised Rotation Regression with Hardness-Aware Curriculum Pseudo Labeling](https://mlsanity.com/paper/2603.21583/) (2026)

## Abstract (arXiv)

Learning about the three-dimensional world from two-dimensional images is a fundamental problem in computer vision. An ideal neural network architecture for such tasks would leverage the fact that objects can be rotated and translated in three dimensions to make predictions about novel images. However, imposing SO(3)-equivariance on two-dimensional inputs is difficult because the group of three-dimensional rotations does not have a natural action on the two-dimensional plane. Specifically, it is possible that an element of SO(3) will rotate an image out of plane. We show that an algorithm that learns a three-dimensional representation of the world from two dimensional images must satisfy certain geometric consistency properties which we formulate as SO(2)-equivariance constraints. We use the induced and restricted representations of SO(2) on SO(3) to construct and classify architectures which satisfy these geometric consistency constraints. We prove that any architecture which respects said consistency constraints can be realized as an instance of our construction. We show that three previously proposed neural architectures for 3D pose prediction are special cases of our construction. We propose a new algorithm that is a learnable generalization of previously considered methods. We test our architecture on three pose predictions task and achieve SOTA results on both the PASCAL3D+ and SYMSOL pose estimation tasks.