mlsanityData to 7 Oct 2026

Equivariant Single View Pose Prediction Via Induced and Restricted Representations

arXiv 2307.03704 ↗First posted 7 Jul 2023cs.CV cs.LG math.GRcited by 6

Not on the map. Why?

Abstract (arXiv)

Learning about the three-dimensional world from two-dimensional images is a fundamental problem in computer vision. An ideal neural network architecture for such tasks would leverage the fact that objects can be rotated and translated in three dimensions to make predictions about novel images. However, imposing SO(3)-equivariance on two-dimensional inputs is difficult because the group of three-dimensional rotations does not have a natural action on the two-dimensional plane. Specifically, it is possible that an element of SO(3) will rotate an image out of plane. We show that an algorithm that learns a three-dimensional representation of the world from two dimensional images must satisfy certain geometric consistency properties which we formulate as SO(2)-equivariance constraints. We use the induced and restricted representations of SO(2) on SO(3) to construct and classify architectures which satisfy these geometric consistency constraints. We prove that any architecture which respects said consistency constraints can be realized as an instance of our construction. We show that three previously proposed neural architectures for 3D pose prediction are special cases of our construction. We propose a new algorithm that is a learnable generalization of previously considered methods. We test our architecture on three pose predictions task and achieve SOTA results on both the PASCAL3D+ and SYMSOL pose estimation tasks.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 47 of its 65 references that are in the map. First posted on arXiv 7 Jul 2023.

Builds on

47 of 65 references.

References · 47

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  4. 2016Group Equivariant Convolutional Networkscited by 1,154
  5. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  6. 2018Self-Attention with Relative Position Representationscited by 1,112
  7. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  8. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  9. 2019General E(2)-Equivariant Steerable CNNscited by 441
  10. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 348
  11. 2016Steerable CNNscited by 298
  12. 2018On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groupscited by 296
35 more
  1. 2017Learning Steerable Filters for Rotation Equivariant CNNscited by 193
  2. 2017A Survey of Structure from Motioncited by 157
  3. 2023Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNscited by 113
  4. 2015Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Viewscited by 94
  5. 2020Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphscited by 90
  6. 2022Object Scene Representation Transformercited by 90
  7. 2023Monocular Depth Estimation using Diffusion Modelscited by 87
  8. 2021Deep Regression on Manifolds: A 3D Rotation Case Studycited by 67
  9. 2018Explorations in Homeomorphic Variational Auto-Encodingcited by 65
  10. 2021Implicit-PDF: Non-Parametric Representation of Probability Distributions on the Rotation Manifoldcited by 65
  11. 2014Viewpoints and Keypointscited by 59
  12. 2019Reconstructing continuous distributions of 3D protein structure from cryo-EM imagescited by 56
  13. 2022Learning Symmetric Embeddings for Equivariant World Modelscited by 54
  14. 2023Invariant Slot Attention: Object Discovery with Slot-Centric Reference Framescited by 51
  15. 2020A Wigner-Eckart Theorem for Group Equivariant Convolution Kernelscited by 46
  16. 2018Deep Directional Statistics: Pose Estimation with Uncertainty Quantificationcited by 45
  17. 2019Dense Depth Estimation in Monocular Endoscopy with Self-supervised Learning Methodscited by 45
  18. 2019Equivariant Multi-View Networkscited by 44
  19. 2020Deep Bingham Networks: Dealing with Uncertainty and Ambiguity in Pose Estimationcited by 43
  20. 2019Spherical Regression: Learning Viewpoints, Surface Normals and 3D Rotations on n-Spherescited by 37
  21. 2022The Surprising Effectiveness of Equivariant Models in Domains with Latent Symmetrycited by 32
  22. 2021Steerable Partial Differential Operators for Equivariant Neural Networkscited by 22
  23. 2022Unified Fourier-based Kernel and Nonlinearity Design for Equivariant Networks on Homogeneous Spacescited by 20
  24. 2018A Mixed Classification-Regression Framework for 3D Pose Estimation from 2D Imagescited by 19
  25. 2022Deconstructing Self-Supervised Monocular Reconstruction: The Design Decisions that Mattercited by 18
  26. 2021Sparse Steerable Convolutions: An Efficient Learning of SE(3)-Equivariant Features for Estimation and Tracking of Object Poses in 3D Spacecited by 17
  27. 2023Image to Sphere: Learning Equivariant Features for Efficient Pose Predictioncited by 17
  28. 2018Intertwiners between Induced Representations (with Applications to the Theory of Equivariant Neural Networks)cited by 16
  29. 2018Cross-Domain 3D Equivariant Image Embeddingscited by 12
  30. 2022Photometric single-view dense 3D reconstruction in endoscopycited by 11
  31. 2021M4Depth: Monocular depth estimation for autonomous vehicles in unseen environmentscited by 9
  32. 2023A Laplace-inspired Distribution on SO(3) for Probabilistic Rotation Estimationcited by 9
  33. 2021Nonlinearities in Steerable SO(2)-Equivariant CNNscited by 3
  34. 2022Image to Icosahedral Projection for SO(3) Object Reasoning from Single-View Imagescited by 3
  35. 2021Probabilistic Regression with Huber Distributionscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 1; 2026: 1.

6 papers.

  1. 2024Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3Dcited by 32
  2. 2023E(2)-Equivariant Graph Planning for Navigationcited by 6
  3. 20243D Equivariant Pose Regression via Direct Wigner-D Harmonics Predictioncited by 4
  4. 2024Equivariant Action Sampling for Reinforcement Learning and Planningcited by 4
  5. 2025Equivariant symmetry-aware head pose estimation for fetal MRIcited by 0
  6. 2026HACMatch Semi-Supervised Rotation Regression with Hardness-Aware Curriculum Pseudo Labelingcited by 0