# Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondence

arXiv: https://arxiv.org/abs/2210.02689 · HTML: https://mlsanity.com/paper/2210.02689/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 91 references that are in the map.
- First posted on arXiv 6 Oct 2022.

## Builds on (most cited first, 30 of 58)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](https://mlsanity.com/paper/2103.13744/) (2021)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction](https://mlsanity.com/paper/2012.03065/) (2020)
- [FlowFormer: A Transformer Architecture for Optical Flow](https://mlsanity.com/paper/2203.16194/) (2022)
- [Space-Time Correspondence as a Contrastive Random Walk](https://mlsanity.com/paper/2006.14613/) (2020)

## Built on by (most cited first, 30 of 32)

- [CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2303.11797/) (2023)
- [PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting](https://mlsanity.com/paper/2410.22128/) (2024)
- [DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalization](https://mlsanity.com/paper/2402.09812/) (2024)
- [Visual Representation Alignment for Multimodal Large Language Models](https://mlsanity.com/paper/2509.07979/) (2025)
- [Unifying Correspondence, Pose and NeRF for Pose-Free Novel View Synthesis from Stereo Pairs](https://mlsanity.com/paper/2312.07246/) (2023)
- [Emergent Temporal Correspondences from Video Diffusion Transformers](https://mlsanity.com/paper/2506.17220/) (2025)
- [Cross-View Completion Models are Zero-shot Correspondence Estimators](https://mlsanity.com/paper/2412.09072/) (2024)
- [Seg4Diff: Unveiling Open-Vocabulary Segmentation in Text-to-Image Diffusion Transformers](https://mlsanity.com/paper/2509.18096/) (2025)
- [Emergent Outlier View Rejection in Visual Geometry Grounded Transformers](https://mlsanity.com/paper/2512.04012/) (2025)
- [C3G: Learning Compact 3D Representations with 2K Gaussians](https://mlsanity.com/paper/2512.04021/) (2025)
- [Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondence](https://mlsanity.com/paper/2403.11120/) (2024)
- [D2USt3R: Enhancing 3D Reconstruction for Dynamic Scenes](https://mlsanity.com/paper/2504.06264/) (2025)
- [Superpixel-informed Implicit Neural Representation for Multi-Dimensional Data](https://mlsanity.com/paper/2411.11356/) (2024)
- [Vid-CamEdit: Video Camera Trajectory Editing with Generative Rendering from Estimated Geometry](https://mlsanity.com/paper/2506.13697/) (2025)
- [Text-Aware Image Restoration with Diffusion Models](https://mlsanity.com/paper/2506.09993/) (2025)
- [S^4M: Boosting Semi-Supervised Instance Segmentation with SAM](https://mlsanity.com/paper/2504.05301/) (2025)
- [CONDA: Condensed Deep Association Learning for Co-Salient Object Detection](https://mlsanity.com/paper/2409.01021/) (2024)
- [Learning 3D Scene Analogies with Neural Contextual Scene Maps](https://mlsanity.com/paper/2503.15897/) (2025)
- [TORA: Topological Representation Alignment for 3D Shape Assembly](https://mlsanity.com/paper/2604.04050/) (2026)
- [Semantic Correspondence: Unified Benchmarking and a Strong Baseline](https://mlsanity.com/paper/2505.18060/) (2025)
- [Entropy-Gradient Grounding: Training-Free Evidence Retrieval in Vision-Language Models](https://mlsanity.com/paper/2604.08456/) (2026)
- [MARCO: Navigating the Unseen Space of Semantic Correspondence](https://mlsanity.com/paper/2604.18267/) (2026)
- [Equivariant Local Reference Frames for Unsupervised Non-rigid Point Cloud Shape Correspondence](https://mlsanity.com/paper/2404.00959/) (2024)
- [No Pose, No Problem in 4D: Feed-Forward Dynamic Gaussians from Unposed Multi-View Videos](https://mlsanity.com/paper/2605.22190/) (2026)
- [Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction](https://mlsanity.com/paper/2605.31595/) (2026)
- [Similarity-Aware Selective State-Space Modeling for Semantic Correspondence](https://mlsanity.com/paper/2509.24318/) (2025)
- [CORAL: Correspondence Alignment for Improved Virtual Try-On](https://mlsanity.com/paper/2602.17636/) (2026)
- [Appearance Matching Adapter for Exemplar-based Semantic Image Synthesis in-the-Wild](https://mlsanity.com/paper/2412.03150/) (2024)
- [BridgeIV: Bridging Customized Image and Video Generation through Test-Time Autoregressive Identity Propagation](https://mlsanity.com/paper/2505.06985/) (2025)
- [Interaction-Centric Knowledge Infusion and Transfer for Open-Vocabulary Scene Graph Generation](https://mlsanity.com/paper/2511.05935/) (2025)

## Abstract (arXiv)

Existing pipelines of semantic correspondence commonly include extracting high-level semantic features for the invariance against intra-class variations and background clutters. This architecture, however, inevitably results in a low-resolution matching field that additionally requires an ad-hoc interpolation process as a post-processing for converting it into a high-resolution one, certainly limiting the overall performance of matching results. To overcome this, inspired by recent success of implicit neural representation, we present a novel method for semantic correspondence, called Neural Matching Field (NeMF). However, complicacy and high-dimensionality of a 4D matching field are the major hindrances, which we propose a cost embedding network to process a coarse cost volume to use as a guidance for establishing high-precision matching field through the following fully-connected network. Nevertheless, learning a high-dimensional matching field remains challenging mainly due to computational complexity, since a naive exhaustive inference would require querying from all pixels in the 4D space to infer pixel-wise correspondences. To overcome this, we propose adequate training and inference procedures, which in the training phase, we randomly sample matching candidates and in the inference phase, we iteratively performs PatchMatch-based inference and coordinate optimization at test time. With these combined, competitive results are attained on several standard benchmarks for semantic correspondence. Code and pre-trained weights are available at https://ku-cvlab.github.io/NeMF/.