# PMatch: Paired Masked Image Modeling for Dense Geometric Matching

arXiv: https://arxiv.org/abs/2303.17342 · HTML: https://mlsanity.com/paper/2303.17342/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 69 references that are in the map.
- First posted on arXiv 30 Mar 2023.

## Builds on (most cited first, 30 of 48)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data](https://mlsanity.com/paper/2111.08897/) (2021)
- [Early Convolutions Help Transformers See Better](https://mlsanity.com/paper/2106.14881/) (2021)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [HPatches: A benchmark and evaluation of handcrafted and learned local descriptors](https://mlsanity.com/paper/1704.05939/) (2017)

## Built on by (most cited first, 29 of 29)

- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [RoMa: Robust Dense Feature Matching](https://mlsanity.com/paper/2305.15404/) (2023)
- [DeDoDe: Detect, Don't Describe -- Describe, Don't Detect for Local Feature Matching](https://mlsanity.com/paper/2308.08479/) (2023)
- [XoFTR: Cross-modal Feature Matching Transformer](https://mlsanity.com/paper/2404.09692/) (2024)
- [Dens3R: A Foundation Model for 3D Geometry Prediction](https://mlsanity.com/paper/2507.16290/) (2025)
- [Tame a Wild Camera: In-the-Wild Monocular Camera Calibration](https://mlsanity.com/paper/2306.10988/) (2023)
- [Reconstructing People, Places, and Cameras](https://mlsanity.com/paper/2412.17806/) (2024)
- [Masked Image Modeling: A Survey](https://mlsanity.com/paper/2408.06687/) (2024)
- [Local Feature Matching Using Deep Learning: A Survey](https://mlsanity.com/paper/2401.17592/) (2024)
- [TopicFM+: Boosting Accuracy and Efficiency of Topic-Assisted Feature Matching](https://mlsanity.com/paper/2307.00485/) (2023)
- [Alligat0R: Pre-Training Through Co-Visibility Segmentation for Relative Camera Pose Regression](https://mlsanity.com/paper/2503.07561/) (2025)
- [UWStereo: A Large Synthetic Dataset for Underwater Stereo Matching](https://mlsanity.com/paper/2409.01782/) (2024)
- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)
- [Low-Light Video Enhancement via Spatial-Temporal Consistent Decomposition](https://mlsanity.com/paper/2405.15660/) (2024)
- [Affine steerers for structured keypoint description](https://mlsanity.com/paper/2408.14186/) (2024)
- [SRPose: Two-view Relative Pose Estimation with Sparse Keypoints](https://mlsanity.com/paper/2407.08199/) (2024)
- [PRISM: PRogressive dependency maxImization for Scale-invariant image Matching](https://mlsanity.com/paper/2408.03598/) (2024)
- [Marginalized Bundle Adjustment: Multi-View Camera Pose from Monocular Depth Estimates](https://mlsanity.com/paper/2602.18906/) (2026)
- [Scalable and Generalizable Correspondence Pruning via Geometry-Consistent Pre-training](https://mlsanity.com/paper/2406.05773/) (2024)
- [XPoint: A Self-Supervised Visual-State-Space based Architecture for Multispectral Image Registration](https://mlsanity.com/paper/2411.07430/) (2024)
- [Region Matters: Efficient and Reliable Region-Aware Visual Place Recognition](https://mlsanity.com/paper/2604.22390/) (2026)
- [Revisit Self-supervised Depth Estimation with Local Structure-from-Motion](https://mlsanity.com/paper/2407.19166/) (2024)
- [Handling Multiple Hypotheses in Coarse-to-Fine Dense Image Matching](https://mlsanity.com/paper/2509.08805/) (2025)
- [Revisiting Cross-View Completion: Self-Supervised Pre-Training via Reconstruction Error Comparison](https://mlsanity.com/paper/2609.01530/) (2026)
- [SoundLoc3D: Invisible 3D Sound Source Localization and Classification Using a Multimodal RGB-D Acoustic Camera](https://mlsanity.com/paper/2412.16861/) (2024)
- [GMatch: A Lightweight, Geometry-Constrained Keypoint Matcher for Zero-Shot 6DoF Pose Estimation in Robotic Grasp Tasks](https://mlsanity.com/paper/2505.16144/) (2025)
- [Low-Light Video Enhancement with An Effective Spatial-Temporal Decomposition Paradigm](https://mlsanity.com/paper/2602.08699/) (2026)
- [REDI-Match: Rotation-Equivariant Distillation for Efficient and Robust Dense Matching](https://mlsanity.com/paper/2606.24330/) (2026)
- [Cross-View Feature Matching: Survey, Benchmarking, and Foundation-Model Perspectives](https://mlsanity.com/paper/2608.11093/) (2026)

## Abstract (arXiv)

Dense geometric matching determines the dense pixel-wise correspondence between a source and support image corresponding to the same 3D structure. Prior works employ an encoder of transformer blocks to correlate the two-frame features. However, existing monocular pretraining tasks, e.g., image classification, and masked image modeling (MIM), can not pretrain the cross-frame module, yielding less optimal performance. To resolve this, we reformulate the MIM from reconstructing a single masked image to reconstructing a pair of masked images, enabling the pretraining of transformer module. Additionally, we incorporate a decoder into pretraining for improved upsampling results. Further, to be robust to the textureless area, we propose a novel cross-frame global matching module (CFGM). Since the most textureless area is planar surfaces, we propose a homography loss to further regularize its learning. Combined together, we achieve the State-of-The-Art (SoTA) performance on geometric matching. Codes and models are available at https://github.com/ShngJZ/PMatch.