mlsanityData to 7 Oct 2026

Grounding Image Matching in 3D with MASt3R

arXiv 2406.09756 ↗First posted 14 Jun 2024cs.CVcited by 960

Not on the map. Why?

Cited by 48 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Image Matching is a core component of all best-performing algorithms and pipelines in 3D vision. Yet despite matching being fundamentally a 3D problem, intrinsically linked to camera pose and scene geometry, it is typically treated as a 2D problem. This makes sense as the goal of matching is to establish correspondences between 2D pixel fields, but also seems like a potentially hazardous choice. In this work, we take a different stance and propose to cast matching as a 3D task with DUSt3R, a recent and powerful 3D reconstruction framework based on Transformers. Based on pointmaps regression, this method displayed impressive robustness in matching views with extreme viewpoint changes, yet with limited accuracy. We aim here to improve the matching capabilities of such an approach while preserving its robustness. We thus propose to augment the DUSt3R network with a new head that outputs dense local features, trained with an additional matching loss. We further address the issue of quadratic complexity of dense matching, which becomes prohibitively slow for downstream applications if not carefully treated. We introduce a fast reciprocal matching scheme that not only accelerates matching by orders of magnitude, but also comes with theoretical guarantees and, lastly, yields improved results. Extensive experiments show that our approach, coined MASt3R, significantly outperforms the state of the art on multiple matching tasks. In particular, it beats the best published methods by 30% (absolute improvement) in VCRE AUC on the extremely challenging Map-free localization dataset.

Key facts. Cited by 960 papers in the map (data through 7 Oct 2026). Builds on 81 of its 114 references that are in the map. First posted on arXiv 14 Jun 2024.

Builds on

81 of 114 references.

References · 81

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2017Billion-scale similarity search with GPUscited by 2,477
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  7. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  8. 2021Vision Transformers for Dense Predictioncited by 1,789
  9. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  10. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  11. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
  12. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
38 more
  1. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 983
  2. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  3. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  4. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 862
  5. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  6. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  7. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  8. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  9. 2023LightGlue: Local Feature Matching at Light Speedcited by 583
  10. 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 556
  11. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  12. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 487
  13. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 421
  14. 2020Virtual KITTI 2cited by 345
  15. 2019D2-Net: A Trainable CNN for Joint Detection and Description of Local Featurescited by 324
  16. 2020Learning to Explore using Active Neural SLAMcited by 308
  17. 2023RoMa: Robust Dense Feature Matchingcited by 296
  18. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 273
  19. 2017HPatches: A benchmark and evaluation of handcrafted and learned local descriptorscited by 261
  20. 2020Image Matching across Wide Baselines: From Paper to Practicecited by 224
  21. 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 223
  22. 2019R2D2: Repeatable and Reliable Detector and Descriptorcited by 206
  23. 2020PatchmatchNet: Learned Multi-View Patchmatch Stereocited by 204
  24. 2021Pixel-Perfect Structure-from-Motion with Featuremetric Refinementcited by 199
  25. 2021COTR: Correspondence Transformer for Matching Across Imagescited by 182
  26. 2022DKM: Dense Kernelized Feature Matching for Geometry Estimationcited by 174
  27. 2022ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformercited by 170
  28. 2022CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flowcited by 165
  29. 2017A Survey of Structure from Motioncited by 157
  30. 2019Cost Volume Pyramid Based Depth Inference for Multi-View Stereocited by 154
  31. 2019Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awarenesscited by 151
  32. 2023PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustmentcited by 146
  33. 2019GLU-Net: Global-Local Universal Network for Dense Flow and Correspondencescited by 139
  34. 2019SOSNet: Second Order Similarity Regularization for Local Descriptor Learningcited by 131
  35. 2020ASLFeat: Learning Local Features of Accurate Shape and Localizationcited by 129
  36. 2022QuadTree Attention for Vision Transformerscited by 129
  37. 2024Cameras as Rays: Pose Estimation via Ray Diffusioncited by 129
  38. 2020Patch2Pix: Epipolar-Guided Pixel-Level Correspondencescited by 128

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 54; 2025: 375; 2026: 531.

960 papers.

Sort:

By area:

  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  2. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  3. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  4. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
  5. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 356
  6. 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 295
  7. 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 292
  8. 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 250
  9. 2024No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Imagescited by 241
  10. 2025AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Viewscited by 234
  11. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 224
  12. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
38 more
  1. 2024MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priorscited by 198
  2. 2025FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Viewscited by 176
  3. 2024Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairscited by 171
  4. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
  5. 2025FoundationStereo: Zero-Shot Stereo Matchingcited by 168
  6. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  7. 2024MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motioncited by 161
  8. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 154
  9. 2025Streaming 4D Visual Geometry Transformercited by 147
  10. 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 139
  11. 2024MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Secondscited by 136
  12. 2025TTT3R: 3D Reconstruction as Test-Time Trainingcited by 122
  13. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 120
  14. 2025Emu3.5: Native Multimodal Models are World Learnerscited by 119
  15. 2025VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequencescited by 107
  16. 2024SLAM3R: Real-Time Dense Scene Reconstruction from Monocular RGB Videoscited by 100
  17. 2025Aether: Geometric-Aware Unified World Modelingcited by 94
  18. 2025FastVGGT: Training-Free Acceleration of Visual Geometry Transformercited by 93
  19. 2025MEt3R: Measuring Multi-View Consistency in Generated Imagescited by 91
  20. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  21. 2025Easi3R: Estimating Disentangled Motion from DUSt3R Without Trainingcited by 88
  22. 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the Worldcited by 88
  23. 2025Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memorycited by 82
  24. 2024Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localizationcited by 81
  25. 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 76
  26. 2025RayZer: A Self-supervised Large View Synthesis Modelcited by 74
  27. 2024Large Spatial Model: End-to-end Unposed Images to Semantic 3Dcited by 73
  28. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 72
  29. 2025Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priorscited by 69
  30. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 66
  31. 2025SpatialLM: Training Large Language Models for Structured Indoor Modelingcited by 63
  32. 2024World-consistent Video Diffusion with Explicit 3D Modelingcited by 62
  33. 2024PF3plat: Pose-Free Feed-Forward 3D Gaussian Splattingcited by 60
  34. 2025UniK3D: Universal Camera Monocular 3D Estimationcited by 60
  35. 2025WorldMirror: Universal 3D World Reconstruction with Any-Prior Promptingcited by 60
  36. 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modelingcited by 59
  37. 2025Bolt3D: Generating 3D Scenes in Secondscited by 57
  38. 2025Evo-0: Vision-Language-Action Model with Implicit Spatial Understandingcited by 55

50 most cited of 960 All 960 (JSON)