# P2-Net: Joint Description and Detection of Local Features for Pixel and Point Matching

arXiv: https://arxiv.org/abs/2103.01055 · HTML: https://mlsanity.com/paper/2103.01055/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 61 references that are in the map.
- First posted on arXiv 1 Mar 2021.

## Builds on (most cited first, 30 of 43)

- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [Circle Loss: A Unified Perspective of Pair Similarity Optimization](https://mlsanity.com/paper/2002.10857/) (2020)
- [3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions](https://mlsanity.com/paper/1603.08182/) (2016)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [HPatches: A benchmark and evaluation of handcrafted and learned local descriptors](https://mlsanity.com/paper/1704.05939/) (2017)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [Deep Global Registration](https://mlsanity.com/paper/2004.11540/) (2020)
- [Working hard to know your neighbor's margins: Local descriptor learning loss](https://mlsanity.com/paper/1705.10872/) (2017)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [PPFNet: Global Context Aware Local Features for Robust 3D Point Matching](https://mlsanity.com/paper/1802.02669/) (2018)
- [D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features](https://mlsanity.com/paper/2003.03164/) (2020)
- [Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC](https://mlsanity.com/paper/2002.12324/) (2020)
- [The Perfect Match: 3D Point Cloud Matching with Smoothed Densities](https://mlsanity.com/paper/1811.06879/) (2018)
- [LF-Net: Learning Local Features from Images](https://mlsanity.com/paper/1805.09662/) (2018)
- [Learning Less is More - 6D Camera Localization via 3D Surface Regression](https://mlsanity.com/paper/1711.10228/) (2017)
- [PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors](https://mlsanity.com/paper/1808.10322/) (2018)
- [SOSNet: Second Order Similarity Regularization for Local Descriptor Learning](https://mlsanity.com/paper/1904.05019/) (2019)
- [ASLFeat: Learning Local Features of Accurate Shape and Localization](https://mlsanity.com/paper/2003.10071/) (2020)
- [Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters](https://mlsanity.com/paper/1904.00889/) (2019)
- [3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration](https://mlsanity.com/paper/1807.09413/) (2018)
- [Hierarchical Scene Coordinate Classification and Regression for Visual Localization](https://mlsanity.com/paper/1909.06216/) (2019)
- [AtLoc: Attention Guided Camera Localization](https://mlsanity.com/paper/1909.03557/) (2019)
- [Learning multiview 3D point cloud registration](https://mlsanity.com/paper/2001.05119/) (2020)
- [USIP: Unsupervised Stable Interest Point Detection from 3D Point Clouds](https://mlsanity.com/paper/1904.00229/) (2019)
- [Learning to Navigate the Energy Landscape](https://mlsanity.com/paper/1603.05772/) (2016)

## Built on by (most cited first, 30 of 42)

- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [FreeReg: Image-to-Point Cloud Registration Leveraging Pretrained Diffusion Models and Monocular Depth Estimators](https://mlsanity.com/paper/2310.03420/) (2023)
- [2D3D-MATR: 2D-3D Matching Transformer for Detection-free Registration between Images and Point Clouds](https://mlsanity.com/paper/2308.05667/) (2023)
- [(LC)2: LiDAR-Camera Loop Constraints For Cross-Modal Place Recognition](https://mlsanity.com/paper/2304.08660/) (2023)
- [3DAxiesPrompts: Unleashing the 3D Spatial Task Capabilities of GPT-4V](https://mlsanity.com/paper/2312.09738/) (2023)
- [CMRNext: Camera to LiDAR Matching in the Wild for Localization and Extrinsic Calibration](https://mlsanity.com/paper/2402.00129/) (2024)
- [3DGS-ReLoc: 3D Gaussian Splatting for Map Representation and Visual ReLocalization](https://mlsanity.com/paper/2403.11367/) (2024)
- [Deep Learning for Visual Localization and Mapping: A Survey](https://mlsanity.com/paper/2308.14039/) (2023)
- [Bridge 2D-3D: Uncertainty-aware Hierarchical Registration Network with Domain Alignment](https://mlsanity.com/paper/2504.01641/) (2025)
- [CoFiI2P: Coarse-to-Fine Correspondences for Image-to-Point Cloud Registration](https://mlsanity.com/paper/2309.14660/) (2023)
- [E2PNet: Event to Point Cloud Registration with Spatio-Temporal Representation Learning](https://mlsanity.com/paper/2311.18433/) (2023)
- [LoMa: Local Feature Matching Revisited](https://mlsanity.com/paper/2604.04931/) (2026)
- [End-to-end 2D-3D Registration between Image and LiDAR Point Cloud for Vehicle Localization](https://mlsanity.com/paper/2306.11346/) (2023)
- [EP2P-Loc: End-to-End 3D Point to 2D Pixel Localization for Large-Scale Visual Localization](https://mlsanity.com/paper/2309.07471/) (2023)
- [CA-I2P: Channel-Adaptive Registration Network with Global Optimal Selection](https://mlsanity.com/paper/2506.21364/) (2025)
- [Diff2I2P: Differentiable Image-to-Point Cloud Registration with Diffusion Prior](https://mlsanity.com/paper/2507.06651/) (2025)
- [Adaptive Agent Selection and Interaction Network for Image-to-point cloud Registration](https://mlsanity.com/paper/2511.05965/) (2025)
- [Contrastive Learning of Features between Images and LiDAR](https://mlsanity.com/paper/2206.12071/) (2022)
- [PointCMC: Cross-Modal Multi-Scale Correspondences Learning for Point Cloud Understanding](https://mlsanity.com/paper/2211.12032/) (2022)
- [ModaLink: Unifying Modalities for Efficient Image-to-PointCloud Place Recognition](https://mlsanity.com/paper/2403.18762/) (2024)
- [Digging Into Self-Supervised Learning of Feature Descriptors](https://mlsanity.com/paper/2110.04773/) (2021)
- [Diff-Reg v1: Diffusion Matching Model for Registration Problem](https://mlsanity.com/paper/2403.19919/) (2024)
- [EdgeRegNet: Edge Feature-based Multimodal Registration Network between Images and LiDAR Point Clouds](https://mlsanity.com/paper/2503.15284/) (2025)
- [EEPNet: Efficient Edge Pixel-based Matching Network for Cross-Modal Dynamic Registration between LiDAR and Camera](https://mlsanity.com/paper/2409.19305/) (2024)
- [Self-Supervised Cross-Modal Learning for Image-to-Point Cloud Registration](https://mlsanity.com/paper/2509.15882/) (2025)
- [GLASS: Geometry-aware Local Alignment and Structure Synchronization Network for 2D-3D Registration](https://mlsanity.com/paper/2603.26262/) (2026)
- [Hg-I2P: Bridging Modalities for Generalizable Image-to-Point-Cloud Registration via Heterogeneous Graphs](https://mlsanity.com/paper/2603.27969/) (2026)
- [Single-Shot Global Localization via Graph-Theoretic Correspondence Matching](https://mlsanity.com/paper/2306.03641/) (2023)
- [TFCT-I2P: Three stream fusion network with color aware transformer for image-to-point cloud registration](https://mlsanity.com/paper/2410.00360/) (2024)
- [EEPNet-V2: Patch-to-Pixel Solution for Efficient Cross-Modal Registration between LiDAR Point Cloud and Camera Image](https://mlsanity.com/paper/2503.15285/) (2025)

## Abstract (arXiv)

Accurately describing and detecting 2D and 3D keypoints is crucial to establishing correspondences across images and point clouds. Despite a plethora of learning-based 2D or 3D local feature descriptors and detectors having been proposed, the derivation of a shared descriptor and joint keypoint detector that directly matches pixels and points remains under-explored by the community. This work takes the initiative to establish fine-grained correspondences between 2D images and 3D point clouds. In order to directly match pixels and points, a dual fully convolutional framework is presented that maps 2D and 3D inputs into a shared latent representation space to simultaneously describe and detect keypoints. Furthermore, an ultra-wide reception mechanism in combination with a novel loss function are designed to mitigate the intrinsic information variations between pixel and point local regions. Extensive experimental results demonstrate that our framework shows competitive performance in fine-grained matching between images and point clouds and achieves state-of-the-art results for the task of indoor visual localization. Our source code will be available at [no-name-for-blind-review].