# Doppelgangers: Learning to Disambiguate Images of Similar Structures

arXiv: https://arxiv.org/abs/2309.02420 · HTML: https://mlsanity.com/paper/2309.02420/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 46 references that are in the map.
- First posted on arXiv 5 Sep 2023.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval](https://mlsanity.com/paper/2004.01804/) (2020)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [Learning to Find Good Correspondences](https://mlsanity.com/paper/1711.05971/) (2017)
- [LF-Net: Learning Local Features from Images](https://mlsanity.com/paper/1805.09662/) (2018)
- [Toward Geometric Deep SLAM](https://mlsanity.com/paper/1707.07410/) (2017)
- [Modeling Perceptual Aliasing in SLAM via Discrete-Continuous Graphical Models](https://mlsanity.com/paper/1810.11692/) (2018)
- [Towers of Babel: Combining Images, Language, and 3D Geometry for Learning Multimodal Vision](https://mlsanity.com/paper/2108.05863/) (2021)
- [Visual Chirality](https://mlsanity.com/paper/2006.09512/) (2020)

## Built on by (most cited first, 30 of 32)

- [Global Structure-from-Motion Revisited](https://mlsanity.com/paper/2407.20219/) (2024)
- [Dense Optical Tracking: Connecting the Dots](https://mlsanity.com/paper/2312.00786/) (2023)
- [MegaScenes: Scene-Level View Synthesis at Scale](https://mlsanity.com/paper/2406.11819/) (2024)
- [AerialMegaDepth: Learning Aerial-Ground Reconstruction and View Synthesis](https://mlsanity.com/paper/2504.13157/) (2025)
- [MP-SfM: Monocular Surface Priors for Robust Structure-from-Motion](https://mlsanity.com/paper/2504.20040/) (2025)
- [Doppelgangers++: Improved Visual Disambiguation with Geometric 3D Features](https://mlsanity.com/paper/2412.05826/) (2024)
- [AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backend](https://mlsanity.com/paper/2511.20343/) (2025)
- [DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusion](https://mlsanity.com/paper/2505.05473/) (2025)
- [FAR: Flexible, Accurate and Robust 6DoF Relative Camera Pose Estimation](https://mlsanity.com/paper/2403.03221/) (2024)
- [LoMa: Local Feature Matching Revisited](https://mlsanity.com/paper/2604.04931/) (2026)
- [Distributed Global Structure-from-Motion with a Deep Front-End](https://mlsanity.com/paper/2311.18801/) (2023)
- [GV-Bench: Benchmarking Local Feature Matching for Geometric Verification of Long-term Loop Closure Detection](https://mlsanity.com/paper/2407.11736/) (2024)
- [Global Structure-from-Motion Meets Feedforward Reconstruction](https://mlsanity.com/paper/2605.26103/) (2026)
- [CoMatcher: Multi-View Collaborative Feature Matching](https://mlsanity.com/paper/2504.01872/) (2025)
- [Extreme Rotation Estimation in the Wild](https://mlsanity.com/paper/2411.07096/) (2024)
- [Same or Not? Enhancing Visual Perception in Vision-Language Models](https://mlsanity.com/paper/2512.23592/) (2025)
- [Long-tail Internet photo reconstruction](https://mlsanity.com/paper/2604.22714/) (2026)
- [Scalable and Generalizable Correspondence Pruning via Geometry-Consistent Pre-training](https://mlsanity.com/paper/2406.05773/) (2024)
- [Metadata-free Georegistration of Ground and Airborne Imagery](https://mlsanity.com/paper/2503.04927/) (2025)
- [Unconstrained Large-scale 3D Reconstruction and Rendering across Altitudes](https://mlsanity.com/paper/2505.00734/) (2025)
- [VidMap: Exploiting Temporal Structure for Video-Based Structure-from-Motion](https://mlsanity.com/paper/2607.27194/) (2026)
- [Global-Aware Edge Prioritization for Pose Graph Initialization](https://mlsanity.com/paper/2602.21963/) (2026)
- [IMC 2024 Methods & Solutions Review](https://mlsanity.com/paper/2407.03172/) (2024)
- [SPIDER: Spatial Image CorresponDence Estimator for Robust Calibration](https://mlsanity.com/paper/2511.17750/) (2025)
- [ArchSym: Detecting 3D-Grounded Architectural Symmetries in the Wild](https://mlsanity.com/paper/2604.22202/) (2026)
- [Honey, I Shrunk the Arc de Triomphe!](https://mlsanity.com/paper/2606.02379/) (2026)
- [Accuracy potential of visual localization exploiting high-end street-level imagery](https://mlsanity.com/paper/2607.24409/) (2026)
- [Robust Global Structure-from-Motion via View Graph Pruning](https://mlsanity.com/paper/2608.22054/) (2026)
- [XDG: Accelerated Visual Disambiguation](https://mlsanity.com/paper/2608.29733/) (2026)
- [HiSfM: Disambiguating Structure-from-Motion via Scaffold-Anchored Hierarchical Reconstruction](https://mlsanity.com/paper/2609.04718/) (2026)

## Abstract (arXiv)

We consider the visual disambiguation task of determining whether a pair of visually similar images depict the same or distinct 3D surfaces (e.g., the same or opposite sides of a symmetric building). Illusory image matches, where two images observe distinct but visually similar 3D surfaces, can be challenging for humans to differentiate, and can also lead 3D reconstruction algorithms to produce erroneous results. We propose a learning-based approach to visual disambiguation, formulating it as a binary classification task on image pairs. To that end, we introduce a new dataset for this problem, Doppelgangers, which includes image pairs of similar structures with ground truth labels. We also design a network architecture that takes the spatial distribution of local keypoints and matches as input, allowing for better reasoning about both local and global cues. Our evaluation shows that our method can distinguish illusory matches in difficult cases, and can be integrated into SfM pipelines to produce correct, disambiguated 3D reconstructions. See our project page for our code, datasets, and more results: http://doppelgangers-3d.github.io/.