# Patch2Pix: Epipolar-Guided Pixel-Level Correspondences

arXiv: https://arxiv.org/abs/2012.01909 · HTML: https://mlsanity.com/paper/2012.01909/

## Key facts

- Cited by 129 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2020.

## Built on by (most cited first, 30 of 129)

- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [Geometric Transformer for Fast and Robust Point Cloud Registration](https://mlsanity.com/paper/2202.06688/) (2022)
- [Pixel-Perfect Structure-from-Motion with Featuremetric Refinement](https://mlsanity.com/paper/2108.08291/) (2021)
- [DKM: Dense Kernelized Feature Matching for Geometry Estimation](https://mlsanity.com/paper/2202.00667/) (2022)
- [CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration](https://mlsanity.com/paper/2110.14076/) (2021)
- [MatchFormer: Interleaving Attention in Transformers for Feature Matching](https://mlsanity.com/paper/2203.09645/) (2022)
- [OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Models](https://mlsanity.com/paper/2301.07673/) (2023)
- [XFeat: Accelerated Features for Lightweight Image Matching](https://mlsanity.com/paper/2404.19174/) (2024)
- [GeoTransformer: Fast and Robust Point Cloud Registration with Geometric Transformer](https://mlsanity.com/paper/2308.03768/) (2023)
- [ClusterGNN: Cluster-based Coarse-to-Fine Graph Neural Network for Efficient Feature Matching](https://mlsanity.com/paper/2204.11700/) (2022)
- [MeshLoc: Mesh-Based Visual Localization](https://mlsanity.com/paper/2207.10762/) (2022)
- [XoFTR: Cross-modal Feature Matching Transformer](https://mlsanity.com/paper/2404.09692/) (2024)
- [Light3R-SfM: Towards Feed-forward Structure-from-Motion](https://mlsanity.com/paper/2501.14914/) (2025)
- [Dens3R: A Foundation Model for 3D Geometry Prediction](https://mlsanity.com/paper/2507.16290/) (2025)
- [IFOR: Iterative Flow Minimization for Robotic Object Rearrangement](https://mlsanity.com/paper/2202.00732/) (2022)
- [MatchAnything: Universal Cross-Modality Image Matching with Large-Scale Pre-Training](https://mlsanity.com/paper/2501.07556/) (2025)
- [AerialMegaDepth: Learning Aerial-Ground Reconstruction and View Synthesis](https://mlsanity.com/paper/2504.13157/) (2025)
- [Learning Super-Features for Image Retrieval](https://mlsanity.com/paper/2201.13182/) (2022)
- [PATS: Patch Area Transportation with Subdivision for Local Feature Matching](https://mlsanity.com/paper/2303.07700/) (2023)
- [TopicFM: Robust and Interpretable Topic-Assisted Feature Matching](https://mlsanity.com/paper/2207.00328/) (2022)
- [The NeRFect Match: Exploring NeRF Features for Visual Localization](https://mlsanity.com/paper/2403.09577/) (2024)
- [3D-Aware Semantic-Guided Generative Model for Human Synthesis](https://mlsanity.com/paper/2112.01422/) (2021)
- [Adaptive Assignment for Geometry Aware Local Feature Matching](https://mlsanity.com/paper/2207.08427/) (2022)
- [Decoupling Makes Weakly Supervised Local Feature Better](https://mlsanity.com/paper/2201.02861/) (2022)
- [SFD2: Semantic-guided Feature Detection and Description](https://mlsanity.com/paper/2304.14845/) (2023)
- [A case for using rotation invariant features in state of the art feature matchers](https://mlsanity.com/paper/2204.10144/) (2022)
- [2D3D-MATR: 2D-3D Matching Transformer for Detection-free Registration between Images and Point Clouds](https://mlsanity.com/paper/2308.05667/) (2023)
- [HSCNet++: Hierarchical Scene Coordinate Classification and Regression for Visual Localization with Transformer](https://mlsanity.com/paper/2305.03595/) (2023)
- [Local Feature Matching Using Deep Learning: A Survey](https://mlsanity.com/paper/2401.17592/) (2024)
- [LASER: LAtent SpacE Rendering for 2D Visual Localization](https://mlsanity.com/paper/2204.00157/) (2022)

## Abstract (arXiv)

The classical matching pipeline used for visual localization typically involves three steps: (i) local feature detection and description, (ii) feature matching, and (iii) outlier rejection. Recently emerged correspondence networks propose to perform those steps inside a single network but suffer from low matching resolution due to the memory bottleneck. In this work, we propose a new perspective to estimate correspondences in a detect-to-refine manner, where we first predict patch-level match proposals and then refine them. We present Patch2Pix, a novel refinement network that refines match proposals by regressing pixel-level matches from the local regions defined by those proposals and jointly rejecting outlier matches with confidence scores. Patch2Pix is weakly supervised to learn correspondences that are consistent with the epipolar geometry of an input image pair. We show that our refinement network significantly improves the performance of correspondence networks on image matching, homography estimation, and localization tasks. In addition, we show that our learned refinement generalizes to fully-supervised methods without re-training, which leads us to state-of-the-art localization performance. The code is available at https://github.com/GrumpyZhou/patch2pix.