# Learning Structure-from-Motion with Graph Attention Networks

arXiv: https://arxiv.org/abs/2308.15984 · HTML: https://mlsanity.com/paper/2308.15984/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 50 references that are in the map.
- First posted on arXiv 30 Aug 2023.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021)
- [NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo](https://mlsanity.com/paper/2109.01129/) (2021)
- [PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustment](https://mlsanity.com/paper/2306.15667/) (2023)
- [DeepSFM: Structure From Motion Via Deep Bundle Adjustment](https://mlsanity.com/paper/1912.09697/) (2019)
- [NeuRoRA: Neural Robust Rotation Averaging](https://mlsanity.com/paper/1912.04485/) (2019)
- [A case for using rotation invariant features in state of the art feature matchers](https://mlsanity.com/paper/2204.10144/) (2022)
- [Algebraic Characterization of Essential Matrices and Their Averaging in Multiview Settings](https://mlsanity.com/paper/1904.02663/) (2019)
- [Deep Permutation Equivariant Structure from Motion](https://mlsanity.com/paper/2104.06703/) (2021)
- [GPSfM: Global Projective SFM Using Algebraic Constraints on Multi-View Fundamental Matrices](https://mlsanity.com/paper/1812.00426/) (2018)

## Built on by (most cited first, 10 of 10)

- [Visual Geometry Grounded Deep Structure From Motion](https://mlsanity.com/paper/2312.04563/) (2023)
- [Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors](https://mlsanity.com/paper/2503.17316/) (2025)
- [Fast Encoder-Based 3D from Casual Videos via Point Track Processing](https://mlsanity.com/paper/2404.07097/) (2024)
- [Affine steerers for structured keypoint description](https://mlsanity.com/paper/2408.14186/) (2024)
- [MuM: Multi-View Masked Image Modeling for 3D Vision](https://mlsanity.com/paper/2511.17309/) (2025)
- [RESfM: Robust Deep Equivariant Structure from Motion](https://mlsanity.com/paper/2404.14280/) (2024)
- [A Taxonomy of Structure from Motion Methods](https://mlsanity.com/paper/2505.15814/) (2025)
- [Maps from Motion (MfM): Generating 2D Semantic Maps from Sparse Multi-view Images](https://mlsanity.com/paper/2411.12620/) (2024)
- [Global-Aware Edge Prioritization for Pose Graph Initialization](https://mlsanity.com/paper/2602.21963/) (2026)
- [Learning Global Camera Poses from Noisy View-Graphs for Structure from Motion](https://mlsanity.com/paper/2609.09491/) (2026)

## Abstract (arXiv)

In this paper we tackle the problem of learning Structure-from-Motion (SfM) through the use of graph attention networks. SfM is a classic computer vision problem that is solved though iterative minimization of reprojection errors, referred to as Bundle Adjustment (BA), starting from a good initialization. In order to obtain a good enough initialization to BA, conventional methods rely on a sequence of sub-problems (such as pairwise pose estimation, pose averaging or triangulation) which provide an initial solution that can then be refined using BA. In this work we replace these sub-problems by learning a model that takes as input the 2D keypoints detected across multiple views, and outputs the corresponding camera poses and 3D keypoint coordinates. Our model takes advantage of graph neural networks to learn SfM-specific primitives, and we show that it can be used for fast inference of the reconstruction for new and unseen sequences. The experimental results show that the proposed model outperforms competing learning-based methods, and challenges COLMAP while having lower runtime. Our code is available at https://github.com/lucasbrynte/gasfm/.