# RAGO: Recurrent Graph Optimizer For Multiple Rotation Averaging

arXiv: https://arxiv.org/abs/2212.07211 · HTML: https://mlsanity.com/paper/2212.07211/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 44 references that are in the map.
- First posted on arXiv 14 Dec 2022.

## Builds on (most cited first, 21 of 21)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [BA-Net: Dense Bundle Adjustment Network](https://mlsanity.com/paper/1806.04807/) (2018)
- [A Survey of Structure from Motion](https://mlsanity.com/paper/1701.08493/) (2017)
- [Solving ill-posed inverse problems using iterative deep neural networks](https://mlsanity.com/paper/1704.04058/) (2017)
- [Learning multiview 3D point cloud registration](https://mlsanity.com/paper/2001.05119/) (2020)
- [Shonan Rotation Averaging: Global Optimality by Surfing SO(p)n](https://mlsanity.com/paper/2008.02737/) (2020)
- [NeuRoRA: Neural Robust Rotation Averaging](https://mlsanity.com/paper/1912.04485/) (2019)
- [Learning Transformation Synchronization](https://mlsanity.com/paper/1901.09458/) (2019)
- [LS-Net: Learning to Solve Nonlinear Least Squares for Monocular Stereo](https://mlsanity.com/paper/1809.02966/) (2018)
- [Message Passing Least Squares Framework and its Application to Rotation Synchronization](https://mlsanity.com/paper/2007.13638/) (2020)
- [Rotation Coordinate Descent for Fast Globally Optimal Rotation Averaging](https://mlsanity.com/paper/2103.08292/) (2021)
- [Learning Iterative Robust Transformation Synchronization](https://mlsanity.com/paper/2111.00728/) (2021)
- [Rotation Averaging in a Split Second: A Primal-Dual Method and a Closed-Form for Cycle Graphs](https://mlsanity.com/paper/2109.08046/) (2021)
- [GSLAM: Initialization-robust Monocular Visual SLAM via Global Structure-from-Motion](https://mlsanity.com/paper/1708.04814/) (2017)

## Built on by (most cited first, 10 of 10)

- [Efficient Detection of Long Consistent Cycles and its Application to Distributed Synchronization](https://mlsanity.com/paper/2407.04260/) (2024)
- [Glob3R: Global Structure-from-Motion with 3D Foundation Models](https://mlsanity.com/paper/2607.09225/) (2026)
- [Adaptive Canonicalization with Application to Invariant Anisotropic Geometric Networks](https://mlsanity.com/paper/2509.24886/) (2025)
- [Incremental Multiview Point Cloud Registration with Two-stage Candidate Retrieval](https://mlsanity.com/paper/2407.07525/) (2024)
- [A Taxonomy of Structure from Motion Methods](https://mlsanity.com/paper/2505.15814/) (2025)
- [Incremental Rotation Averaging Revisited](https://mlsanity.com/paper/2309.16924/) (2023)
- [Matching Distance and Geometric Distribution Aided Learning Multiview Point Cloud Registration](https://mlsanity.com/paper/2505.03692/) (2025)
- [EAR-Net: Pursuing End-to-End Absolute Rotations from Multi-View Images](https://mlsanity.com/paper/2310.10051/) (2023)
- [PRAGO: Differentiable Multi-View Pose Optimization From Objectness Detections](https://mlsanity.com/paper/2403.08586/) (2024)
- [Multiple Rotation Averaging with Constrained Reweighting Deep Matrix Factorization](https://mlsanity.com/paper/2409.09790/) (2024)

## Abstract (arXiv)

This paper proposes a deep recurrent Rotation Averaging Graph Optimizer (RAGO) for Multiple Rotation Averaging (MRA). Conventional optimization-based methods usually fail to produce accurate results due to corrupted and noisy relative measurements. Recent learning-based approaches regard MRA as a regression problem, while these methods are sensitive to initialization due to the gauge freedom problem. To handle these problems, we propose a learnable iterative graph optimizer minimizing a gauge-invariant cost function with an edge rectification strategy to mitigate the effect of inaccurate measurements. Our graph optimizer iteratively refines the global camera rotations by minimizing each node's single rotation objective function. Besides, our approach iteratively rectifies relative rotations to make them more consistent with the current camera orientations and observed relative rotations. Furthermore, we employ a gated recurrent unit to improve the result by tracing the temporal information of the cost graph. Our framework is a real-time learning-to-optimize rotation averaging graph optimizer with a tiny size deployed for real-world applications. RAGO outperforms previous traditional and deep methods on real-world and synthetic datasets. The code is available at https://github.com/sfu-gruvi-3dv/RAGO