# Computing CNN Loss and Gradients for Pose Estimation with Riemannian Geometry

arXiv: https://arxiv.org/abs/1805.01026 · HTML: https://mlsanity.com/paper/1805.01026/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2018.

## Built on by (most cited first, 11 of 11)

- [Riemannian Score-Based Generative Modelling](https://mlsanity.com/paper/2202.02763/) (2022)
- [Projective Manifold Gradient Layer for Deep Rotation Regression](https://mlsanity.com/paper/2110.11657/) (2021)
- [SVoRT: Iterative Transformer for Slice-to-Volume Registration in Fetal Brain MRI](https://mlsanity.com/paper/2206.10802/) (2022)
- [The Impact of Machine Learning on 2D/3D Registration for Image-guided Interventions: A Systematic Review and Perspective](https://mlsanity.com/paper/2108.02238/) (2021)
- [Fully Convolutional Slice-to-Volume Reconstruction for Single-Stack MRI](https://mlsanity.com/paper/2312.03102/) (2023)
- [Long-term Dependency for 3D Reconstruction of Freehand Ultrasound Without External Tracker](https://mlsanity.com/paper/2310.10248/) (2023)
- [tensorflow-riemopt: A Library for Optimization on Riemannian Manifolds](https://mlsanity.com/paper/2105.13921/) (2021)
- [Meta-learning Slice-to-Volume Reconstruction in Fetal Brain MRI using Implicit Neural Representations](https://mlsanity.com/paper/2505.09565/) (2025)
- [Intraoperative 2D/3D Registration via Spherical Similarity Learning and Differentiable Levenberg-Marquardt Optimization](https://mlsanity.com/paper/2509.06890/) (2025)
- [T-MASK: Temporal Masking for Probing Foundation Models across Camera Views in Driver Monitoring](https://mlsanity.com/paper/2508.16207/) (2025)
- [SPARC: Slice-to-volume Pipeline for Automated Reconstruction of gated 3D+time fetal Cardiac MRI](https://mlsanity.com/paper/2608.18616/) (2026)

## Abstract (arXiv)

Pose estimation, i.e. predicting a 3D rigid transformation with respect to a fixed co-ordinate frame in, SE(3), is an omnipresent problem in medical image analysis with applications such as: image rigid registration, anatomical standard plane detection, tracking and device/camera pose estimation. Deep learning methods often parameterise a pose with a representation that separates rotation and translation. As commonly available frameworks do not provide means to calculate loss on a manifold, regression is usually performed using the L2-norm independently on the rotation's and the translation's parameterisations, which is a metric for linear spaces that does not take into account the Lie group structure of SE(3). In this paper, we propose a general Riemannian formulation of the pose estimation problem. We propose to train the CNN directly on SE(3) equipped with a left-invariant Riemannian metric, coupling the prediction of the translation and rotation defining the pose. At each training step, the ground truth and predicted pose are elements of the manifold, where the loss is calculated as the Riemannian geodesic distance. We then compute the optimisation direction by back-propagating the gradient with respect to the predicted pose on the tangent space of the manifold SE(3) and update the network weights. We thoroughly evaluate the effectiveness of our loss function by comparing its performance with popular and most commonly used existing methods, on tasks such as image-based localisation and intensity-based 2D/3D registration. We also show that hyper-parameters, used in our loss function to weight the contribution between rotations and translations, can be intrinsically calculated from the dataset to achieve greater performance margins.