mlsanityData to 7 Oct 2026

Projective Manifold Gradient Layer for Deep Rotation Regression

arXiv 2110.11657 ↗First posted 22 Oct 2021cs.CVcited by 31

Not on the map. Why?

Abstract (arXiv)

Regressing rotations on SO(3) manifold using deep neural networks is an important yet unsolved problem. The gap between the Euclidean network output space and the non-Euclidean SO(3) manifold imposes a severe challenge for neural network learning in both forward and backward passes. While several works have proposed different regression-friendly rotation representations, very few works have been devoted to improving the gradient backpropagating in the backward pass. In this paper, we propose a manifold-aware gradient that directly backpropagates into deep network weights. Leveraging Riemannian optimization to construct a novel projective gradient, our proposed regularized projective manifold gradient (RPMG) method helps networks achieve new state-of-the-art performance in a variety of rotation estimation tasks. Our proposed gradient layer can also be applied to other smooth manifolds such as the unit sphere. Our project page is at https://jychen18.github.io/RPMG.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 31 of its 48 references that are in the map. First posted on arXiv 22 Oct 2021.

Builds on

31 of 48 references.

References · 31

  1. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  2. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  3. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  4. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  5. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  6. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 490
  7. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 352
  8. 2017Geometric Loss Functions for Camera Pose Regression with Deep Learningcited by 224
  9. 2016DeMoN: Depth and Motion Network for Learning Monocular Stereocited by 200
  10. 2016Image-based localization using LSTMs for structured feature correlationcited by 136
  11. 2015Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Viewscited by 94
  12. 2020An Analysis of SVD for Deep Rotation Estimationcited by 85
19 more
  1. 2021Equivariant Point Network for 3D Point Cloud Analysiscited by 81
  2. 2020Self6D: Self-Supervised Monocular 6D Object Pose Estimationcited by 79
  3. 2020Learning multiview 3D point cloud registrationcited by 78
  4. 2021Deep Regression on Manifolds: A 3D Rotation Case Studycited by 67
  5. 2018Deep Part Induction from Articulated Object Pairscited by 66
  6. 2016Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifoldscited by 64
  7. 2014Viewpoints and Keypointscited by 59
  8. 2021MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronizationcited by 46
  9. 2020Deep Bingham Networks: Dealing with Uncertainty and Ambiguity in Pose Estimationcited by 43
  10. 2019Quaternion Equivariant Capsule Networks for 3D Point Cloudscited by 41
  11. 2021Tangent Space Backpropagation for 3D Transformation Groupscited by 40
  12. 2019Spherical Regression: Learning Viewpoints, Surface Normals and 3D Rotations on n-Spherescited by 37
  13. 2018Deep-6DPose: Recovering 6D Object Pose from a Single RGB Imagecited by 26
  14. 2019Probabilistic Permutation Synchronization using the Riemannian Structure of the Birkhoff Polytopecited by 26
  15. 20206D Camera Relocalization in Ambiguous Scenes via Continuous Multimodal Inferencecited by 23
  16. 2018Bayesian Pose Graph Optimization via Bingham Distributions and Tempered Geodesic MCMCcited by 19
  17. 2020Robust Neural Routing Through Space Partitions for Camera Relocalization in Dynamic Indoor Environmentscited by 19
  18. 2018Computing CNN Loss and Gradients for Pose Estimation with Riemannian Geometrycited by 11
  19. 2018Occlusion Resistant Object Rotation Regression from Point Cloud Segmentscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 7; 2024: 7; 2025: 8; 2026: 5.

31 papers.

Sort:

By area:

  1. 2025RayZer: A Self-supervised Large View Synthesis Modelcited by 74
  2. 2024Learning with 3D rotations, a hitchhiker's guide to SO(3)cited by 39
  3. 2024NRDF: Neural Riemannian Distance Fields for Learning Articulated Pose Priorscited by 27
  4. 2023GAMMA: Graspability-Aware Mobile MAnipulation Policy Learning based on Online Grasping Pose Fusioncited by 21
  5. 2022FisherMatch: Semi-Supervised Rotation Regression via Entropy-based Filteringcited by 16
  6. 2023Non-Rigid Shape Registration via Deep Functional Maps Priorcited by 12
  7. 2024SRIF: Semantic Shape Registration Empowered by Diffusion-based Image Morphing and Flow Estimationcited by 12
  8. 2023A Laplace-inspired Distribution on SO(3) for Probabilistic Rotation Estimationcited by 9
  9. 2023Learning Generalizable Tool-use Skills through Trajectory Generationcited by 6
  10. 2025UP-SLAM: Adaptively Structured Gaussian SLAM with Uncertainty Prediction in Dynamic Environmentscited by 6
  11. 2025Geometric Neural Distance Fields for Learning Human Motion Priorscited by 6
  12. 2023Towards Robust Probabilistic Modeling on SO(3) via Rotation Laplace Distributioncited by 5
19 more
  1. 20243D Equivariant Pose Regression via Direct Wigner-D Harmonics Predictioncited by 4
  2. 2023Learning to Localize in Unseen Scenes with Relative Pose Regressorscited by 3
  3. 2024Causal Transformer for Fusion and Pose Estimation in Deep Visual Inertial Odometrycited by 3
  4. 2025Forecasting Continuous Non-Conservative Dynamical Systems in SO(3)cited by 3
  5. 2022TetraSphere: A Neural Descriptor for O(3)-Invariant Point Cloud Analysiscited by 2
  6. 2024Alignist: CAD-Informed Orientation Distribution Estimation by Fusing Shape and Correspondencescited by 2
  7. 2025Continuous-Time SO(3) Forecasting with Savitzky--Golay Neural Controlled Differential Equationscited by 2
  8. 2025Category-Level Object Shape and Pose Estimation in Less Than a Millisecondcited by 2
  9. 2026Parallelised Differentiable Straightest Geodesics for 3D Meshescited by 2
  10. 2022Deep Projective Rotation Estimation through Relative Supervisioncited by 1
  11. 2023A survey of manifold learning and its applications for multimediacited by 1
  12. 2026PDF-HR: Pose Distance Fields for Humanoid Robotscited by 1
  13. 2026Cov2Pose: Leveraging Spatial Covariance for Direct Manifold-aware 6-DoF Object Pose Estimationcited by 1
  14. 2022Changes from Classical Statistics to Modern Statistics and Data Sciencecited by 0
  15. 2024Deep Extrinsic Manifold Representation for Vision Taskscited by 0
  16. 2025NFR: Neural Feature-Guided Non-Rigid Shape Registrationcited by 0
  17. 2025RA-NeRF: Robust Neural Radiance Field Reconstruction with Accurate Camera Pose Estimation under Complex Trajectoriescited by 0
  18. 2026DAP-Pose: Deep Temporal Alignment and Physics-aware Cross-modal Sensor Fusion for Robust Pose Estimationcited by 0
  19. 2026A Geometry-Driven, Framework-Agnostic Optimization for Object Pose Estimationcited by 0