mlsanityData to 14 Oct 2026

Learning with 3D rotations, a hitchhiker's guide to SO(3)

arXiv 2404.11735 ↗First posted 17 Apr 2024cs.LG cs.CV cs.ROcited by 42

Not on the map. Why?

Abstract (arXiv)

Many settings in machine learning require the selection of a rotation representation. However, choosing a suitable representation from the many available options is challenging. This paper acts as a survey and guide through rotation representations. We walk through their properties that harm or benefit deep learning with gradient-based optimization. By consolidating insights from rotation-based learning, we provide a comprehensive overview of learning functions with rotation representations. We provide guidance on selecting representations based on whether rotations are in the model's input or output and whether the data primarily comprises small angles.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Builds on 13 of its 30 references that are in the map. First posted on arXiv 17 Apr 2024.

Builds on

13 of 30 references.

References · 13

  1. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  2. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 775
  3. 2017Sharp Minima Can Generalize For Deep Netscited by 466
  4. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 355
  5. 2021Modern Koopman Theory for Dynamical Systemscited by 276
  6. 2023Understanding plasticity in neural networkscited by 174
  7. 2021Position Information in Transformers: An Overviewcited by 103
  8. 2020An Analysis of SVD for Deep Rotation Estimationcited by 86
  9. 2021Deep Regression on Manifolds: A 3D Rotation Case Studycited by 67
  10. 2019Approximation Capabilities of Neural ODEs and Invertible Residual Networkscited by 56
  11. 2021A Loss Curvature Perspective on Training Instability in Deep Learningcited by 40
  12. 2021Projective Manifold Gradient Layer for Deep Rotation Regressioncited by 31
1 more
  1. 2023Loss it right: Euclidean and Riemannian Metrics in Learning-based Visual Odometrycited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 21; 2026: 13.

42 papers.

Sort:

By area:

  1. 2024From Imitation to Refinement -- Residual RL for Precise Assemblycited by 90
  2. 2024Learning Robotic Manipulation Policies from Point Clouds with Conditional Flow Matchingcited by 65
  3. 2025Towards Human-level Intelligence via Human-like Whole-Body Manipulationcited by 13
  4. 2025Flexible MOF Generation with Torsion-Aware Flow Matchingcited by 10
  5. 2025Steerable Scene Generation with Post Training and Inference-Time Searchcited by 9
  6. 2025Mind to Hand: Purposeful Robotic Control via Embodied Reasoningcited by 8
  7. 2024Reconstructing Animals and the Wildcited by 7
  8. 2026Reconstruction by Generation: 3D Multi-Object Scene Reconstruction from Sparse Observationscited by 7
  9. 2024Whole-body End-Effector Pose Trackingcited by 6
  10. 2024AutoURDF: Unsupervised Robot Modeling from Point Cloud Frames Using Cluster Registrationcited by 5
  11. 2025Differentiable Simulation of Hard Contacts with Soft Gradients for Learning and Controlcited by 5
  12. 20243D Equivariant Pose Regression via Direct Wigner-D Harmonics Predictioncited by 4
30 more
  1. 2025Scalable Non-Equivariant 3D Molecule Generation via Rotational Alignmentcited by 3
  2. 2025Forecasting Continuous Non-Conservative Dynamical Systems in SO(3)cited by 3
  3. 2026Geometric Flow Matching for Molecular Conformation Generation via Manifold Decompositioncited by 3
  4. 2025MatrixNet: Learning over symmetry groups using learned group representationscited by 2
  5. 2025Continuous-Time SO(3) Forecasting with Savitzky--Golay Neural Controlled Differential Equationscited by 2
  6. 2025A Primer on SO(3) Action Representations in Deep Reinforcement Learningcited by 2
  7. 2025SPEAR-1: Scaling Beyond Robot Demonstrations via 3D Understandingcited by 2
  8. 2025VLD: Visual Language Goal Distance for Reinforcement Learning Navigationcited by 2
  9. 2024Reinforcement Learning with Lie Group Orientations for Roboticscited by 1
  10. 2025Multi-Modal Gesture Recognition from Video and Surgical Tool Pose Information via Motion Invariantscited by 1
  11. 2025Learning Agile Gate Traversal via Analytical Optimal Policy Gradientcited by 1
  12. 2026Cov2Pose: Leveraging Spatial Covariance for Direct Manifold-aware 6-DoF Object Pose Estimationcited by 1
  13. 2026Learning Action-Conditional and Object-Centric Gaussian Splatting World Models for Rigid Objectscited by 1
  14. 2026EquiFusion: Kinematics-Agnostic Human Motion Prediction via Equivariant Latent Diffusioncited by 1
  15. 2024Cross-Attention Based Influence Model for Manual and Nonmanual Sign Language Analysiscited by 0
  16. 2025A comparison of visual representations for real-world reinforcement learning in the context of vacuum grippingcited by 0
  17. 2025FloPE: Flower Pose Estimation for Precision Pollinationcited by 0
  18. 2025Rapid patient-specific neural networks for intraoperative X-ray to volume registrationcited by 0
  19. 2025T-MASK: Temporal Masking for Probing Foundation Models across Camera Views in Driver Monitoringcited by 0
  20. 2025Learning and Optimization with 3D Orientationscited by 0
  21. 2025Towards Foundation Models for Cryo-ET Subtomogram Analysiscited by 0
  22. 2025Equivariant symmetry-aware head pose estimation for fetal MRIcited by 0
  23. 2026Yolo-Key-6D: Single Stage Monocular 6D Pose Estimation with Keypoint Enhancementscited by 0
  24. 2026Training Without Orthogonalization, Inference With SVD: A Gradient Analysis of Rotation Representationscited by 0
  25. 2026Beyond Binary: Sim-to-Real Dexterous Manipulation with Physics-Grounded Contact Representationcited by 0
  26. 2026OctoSense: Self-Supervised Learning for Multimodal Robot Perceptioncited by 0
  27. 2026ELMP: Efficient Learning for Motion Planning via Analytical Policy Gradientscited by 0
  28. 2026Revisiting Euler-Angle Regression with Kolmogorov-Arnold Networkscited by 0
  29. 2026Predicting build orientation for SLM dental parts: a comparison of rotation representations and direct vector regressioncited by 0
  30. 2026PART: Learning 3D Part Assembly and Retrieval with Transformerscited by 0