A Primer on SO(3) Action Representations in Deep Reinforcement Learning
On the map: Deep reinforcement learning methods → Deep RL algorithms and representations
Abstract (arXiv)
Many robotic control tasks require policies to act on orientations, yet the geometry of SO(3) makes this nontrivial. Because SO(3) admits no global, smooth, minimal parameterization, common representations such as Euler angles, quaternions, rotation matrices, and Lie algebra coordinates introduce distinct constraints and failure modes. While these trade-offs are well studied for supervised learning, their implications for actions in reinforcement learning remain unclear. We systematically evaluate SO(3) action representations across three standard continuous control algorithms, PPO, SAC, and TD3, under dense and sparse rewards. We compare how representations shape exploration, interact with entropy regularization, and affect training stability through empirical studies and analyze the implications of different projections for obtaining valid rotations from Euclidean network outputs. Across a suite of robotics benchmarks, we quantify the practical impact of these choices and distill simple, implementation-ready guidelines for selecting and using rotation actions. Our results highlight that representation-induced geometry strongly influences exploration and optimization and show that representing actions as tangent vectors in the local frame yields the most reliable results across algorithms. The project webpage and code are available at amacati.github.io/so3_primer.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 16 of its 31 references that are in the map. First posted on arXiv 13 Oct 2025.
Builds on
16 of 31 references.
References · 16
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
- 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
- 2017Hindsight Experience Replaycited by 1,136
- 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
- 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
- 2017Parameter Space Noise for Explorationcited by 161
- 2018A micro Lie theory for state estimation in roboticscited by 124
- 2021Deep Regression on Manifolds: A 3D Rotation Case Studycited by 67
- 2021Autonomous Drone Racing with Deep Reinforcement Learningcited by 60
4 more
- 2021safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Roboticscited by 50
- 2024Learning with 3D rotations, a hitchhiker's guide to SO(3)cited by 42
- 2022Geometric Reinforcement Learning For Robotic Manipulationcited by 5
- 2024Reinforcement Learning with Lie Group Orientations for Roboticscited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2022Implicit Kinematic Policies: Unifying Joint and Cartesian Action Spaces in End-to-End Robot Learning
- 2024Unsupervised Learning of Effective Actions in Robotics
- 2026SAVGO: Learning State-Action Value Geometry with Cosine Similarity for Continuous Control
- 2024Geometric Fabrics: a Safe Guiding Medium for Policy Learning
- 2026Factorized Spectral Representations for Reinforcement Learning
- 2026PoLAR: Factorizing Extent and Mode in Latent Actions for Robot Policy Learning
- 2025Geometry of Neural Reinforcement Learning in Continuous State and Action Spaces
- 2026Matrix-Space Reinforcement Learning for Reusing Local Transition Geometry