# Learning Camera Localization via Dense Scene Matching

arXiv: https://arxiv.org/abs/2103.16792 · HTML: https://mlsanity.com/paper/2103.16792/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 74 references that are in the map.
- First posted on arXiv 31 Mar 2021.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [Learning with Average Precision: Training Image Retrieval with a Listwise Loss](https://mlsanity.com/paper/1906.07589/) (2019)
- [Understanding the Limitations of CNN-based Absolute Camera Pose Regression](https://mlsanity.com/paper/1903.07504/) (2019)
- [Learning to Find Good Correspondences](https://mlsanity.com/paper/1711.05971/) (2017)
- [Geometry-Aware Learning of Maps for Camera Localization](https://mlsanity.com/paper/1712.03342/) (2017)
- [Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches](https://mlsanity.com/paper/1510.05970/) (2015)
- [Deep Image Retrieval: Learning global representations for image search](https://mlsanity.com/paper/1604.01325/) (2016)
- [Learning Less is More - 6D Camera Localization via 3D Surface Regression](https://mlsanity.com/paper/1711.10228/) (2017)
- [Image-based localization using LSTMs for structured feature correlation](https://mlsanity.com/paper/1611.07890/) (2016)
- [Modelling Uncertainty in Deep Learning for Camera Relocalization](https://mlsanity.com/paper/1509.05909/) (2015)
- [Hierarchical Scene Coordinate Classification and Regression for Visual Localization](https://mlsanity.com/paper/1909.06216/) (2019)
- [Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Network](https://mlsanity.com/paper/1707.09733/) (2017)

## Built on by (most cited first, 28 of 28)

- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [MatchFormer: Interleaving Attention in Transformers for Feature Matching](https://mlsanity.com/paper/2203.09645/) (2022)
- [Learning to Match Features with Seeded Graph Matching Network](https://mlsanity.com/paper/2108.08771/) (2021)
- [Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localization](https://mlsanity.com/paper/2412.08376/) (2024)
- [Visual Localization via Few-Shot Scene Region Classification](https://mlsanity.com/paper/2208.06933/) (2022)
- [The NeRFect Match: Exploring NeRF Features for Visual Localization](https://mlsanity.com/paper/2403.09577/) (2024)
- [NeRF-Loc: Visual Localization with Conditional Neural Radiance Field](https://mlsanity.com/paper/2304.07979/) (2023)
- [Investigating the Role of Image Retrieval for Visual Localization -- An exhaustive benchmark](https://mlsanity.com/paper/2205.15761/) (2022)
- [NeuMap: Neural Coordinate Mapping by Auto-Transdecoder for Camera Localization](https://mlsanity.com/paper/2211.11177/) (2022)
- [SAIL-Recon: Large SfM by Augmenting Scene Regression with Localization](https://mlsanity.com/paper/2508.17972/) (2025)
- [SACReg: Scene-Agnostic Coordinate Regression for Visual Localization](https://mlsanity.com/paper/2307.11702/) (2023)
- [Deep Learning for Visual Localization and Mapping: A Survey](https://mlsanity.com/paper/2308.14039/) (2023)
- [Leveraging Neural Radiance Fields for Uncertainty-Aware Visual Localization](https://mlsanity.com/paper/2310.06984/) (2023)
- [Learning to Produce Semi-dense Correspondences for Visual Localization](https://mlsanity.com/paper/2402.08359/) (2024)
- [RTG-SLAM: Real-time 3D Reconstruction at Scale using Gaussian Splatting](https://mlsanity.com/paper/2404.19706/) (2024)
- [A Guide to Structureless Visual Localization](https://mlsanity.com/paper/2504.17636/) (2025)
- [Lazy Visual Localization via Motion Averaging](https://mlsanity.com/paper/2307.09981/) (2023)
- [GSVisLoc: Generalizable Visual Localization for Gaussian Splatting Scene Representations](https://mlsanity.com/paper/2508.18242/) (2025)
- [Video Understanding: From Geometry and Semantics to Unified Models](https://mlsanity.com/paper/2603.17840/) (2026)
- [Learning Where to Look: Self-supervised Viewpoint Selection for Active Localization using Geometrical Information](https://mlsanity.com/paper/2407.15593/) (2024)
- [ActLoc: Learning to Localize on the Move via Active Viewpoint Selection](https://mlsanity.com/paper/2508.20981/) (2025)
- [Enhancing Scene Coordinate Regression with Efficient Keypoint Detection and Sequential Information](https://mlsanity.com/paper/2412.06488/) (2024)
- [GRLoc: Geometric Representation Regression for Visual Localization](https://mlsanity.com/paper/2511.13864/) (2025)
- [Graph Attention Network for Camera Relocalization on Dynamic Scenes](https://mlsanity.com/paper/2209.15056/) (2022)
- [SGL: Structure Guidance Learning for Camera Localization](https://mlsanity.com/paper/2304.05571/) (2023)
- [PRAM: Place Recognition Anywhere Model for Efficient Visual Localization](https://mlsanity.com/paper/2404.07785/) (2024)
- [Multiview Image-Based Localization](https://mlsanity.com/paper/2503.23577/) (2025)
- [MultiLoc: Look Around As You Localize for Fast And Robust Visual Re-localization](https://mlsanity.com/paper/2603.27170/) (2026)

## Abstract (arXiv)

Camera localization aims to estimate 6 DoF camera poses from RGB images. Traditional methods detect and match interest points between a query image and a pre-built 3D model. Recent learning-based approaches encode scene structures into a specific convolutional neural network (CNN) and thus are able to predict dense coordinates from RGB images. However, most of them require re-training or re-adaption for a new scene and have difficulties in handling large-scale scenes due to limited network capacity. We present a new method for scene agnostic camera localization using dense scene matching (DSM), where a cost volume is constructed between a query image and a scene. The cost volume and the corresponding coordinates are processed by a CNN to predict dense coordinates. Camera poses can then be solved by PnP algorithms. In addition, our method can be extended to temporal domain, which leads to extra performance boost during testing time. Our scene-agnostic approach achieves comparable accuracy as the existing scene-specific approaches, such as KFNet, on the 7scenes and Cambridge benchmark. This approach also remarkably outperforms state-of-the-art scene-agnostic dense coordinate regression network SANet. The Code is available at https://github.com/Tangshitao/Dense-Scene-Matching.