# RESfM: Robust Deep Equivariant Structure from Motion

arXiv: https://arxiv.org/abs/2404.14280 · HTML: https://mlsanity.com/paper/2404.14280/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 53 references that are in the map.
- First posted on arXiv 22 Apr 2024.

## Builds on (most cited first, 30 of 31)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [Visual Geometry Grounded Deep Structure From Motion](https://mlsanity.com/paper/2312.04563/) (2023)
- [Pixel-Perfect Structure-from-Motion with Featuremetric Refinement](https://mlsanity.com/paper/2108.08291/) (2021)
- [Learning to Find Good Correspondences](https://mlsanity.com/paper/1711.05971/) (2017)
- [A Survey of Structure from Motion](https://mlsanity.com/paper/1701.08493/) (2017)
- [Learning Two-View Correspondences and Geometry Using Order-Aware Network](https://mlsanity.com/paper/1908.04964/) (2019)
- [PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustment](https://mlsanity.com/paper/2306.15667/) (2023)
- [Cameras as Rays: Pose Estimation via Ray Diffusion](https://mlsanity.com/paper/2402.14817/) (2024)
- [Map-free Visual Relocalization: Metric Pose Relative to a Single Image](https://mlsanity.com/paper/2210.05494/) (2022)
- [RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wild](https://mlsanity.com/paper/2208.05963/) (2022)
- [RelPose++: Recovering 6D Poses from Sparse-view Observations](https://mlsanity.com/paper/2305.04926/) (2023)
- [Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Network](https://mlsanity.com/paper/1707.09733/) (2017)
- [Detector-Free Structure from Motion](https://mlsanity.com/paper/2306.15669/) (2023)
- [Deep Models of Interactions Across Sets](https://mlsanity.com/paper/1803.02879/) (2018)
- [SparsePose: Sparse-View Camera Pose Regression and Refinement](https://mlsanity.com/paper/2211.16991/) (2022)
- [Progressive Correspondence Pruning by Consensus Learning](https://mlsanity.com/paper/2101.00591/) (2021)
- [Extreme Rotation Estimation using Dense Correlation Volumes](https://mlsanity.com/paper/2104.13530/) (2021)
- [Robust Camera Location Estimation by Convex Programming](https://mlsanity.com/paper/1412.0165/) (2014)
- [The 8-Point Algorithm as an Inductive Bias for Relative Pose Prediction by ViTs](https://mlsanity.com/paper/2208.08988/) (2022)
- [Algebraic Characterization of Essential Matrices and Their Averaging in Multiview Settings](https://mlsanity.com/paper/1904.02663/) (2019)
- [Deep Permutation Equivariant Structure from Motion](https://mlsanity.com/paper/2104.06703/) (2021)
- [Learning Structure-from-Motion with Graph Attention Networks](https://mlsanity.com/paper/2308.15984/) (2023)

## Built on by (most cited first, 2 of 2)

- [A Taxonomy of Structure from Motion Methods](https://mlsanity.com/paper/2505.15814/) (2025)
- [Learning Global Camera Poses from Noisy View-Graphs for Structure from Motion](https://mlsanity.com/paper/2609.09491/) (2026)

## Abstract (arXiv)

Multiview Structure from Motion is a fundamental and challenging computer vision problem. A recent deep-based approach utilized matrix equivariant architectures for simultaneous recovery of camera pose and 3D scene structure from large image collections. That work, however, made the unrealistic assumption that the point tracks given as input are almost clean of outliers. Here, we propose an architecture suited to dealing with outliers by adding a multiview inlier/outlier classification module that respects the model equivariance and by utilizing a robust bundle adjustment step. Experiments demonstrate that our method can be applied successfully in realistic settings that include large image collections and point tracks extracted with common heuristics that include many outliers, achieving state-of-the-art accuracies in almost all runs, superior to existing deep-based methods and on-par with leading classical (non-deep) sequential and global methods.