# Efficient Neighbourhood Consensus Networks via Submanifold Sparse Convolutions

arXiv: https://arxiv.org/abs/2004.10566 · HTML: https://mlsanity.com/paper/2004.10566/

## Key facts

- Cited by 93 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Apr 2020.

## Built on by (most cited first, 30 of 93)

- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [Pixel-Perfect Structure-from-Motion with Featuremetric Refinement](https://mlsanity.com/paper/2108.08291/) (2021)
- [Hypercorrelation Squeeze for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01538/) (2021)
- [DKM: Dense Kernelized Feature Matching for Geometry Estimation](https://mlsanity.com/paper/2202.00667/) (2022)
- [ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer](https://mlsanity.com/paper/2208.14201/) (2022)
- [AiATrack: Attention in Attention for Transformer Visual Tracking](https://mlsanity.com/paper/2207.09603/) (2022)
- [MatchFormer: Interleaving Attention in Transformers for Feature Matching](https://mlsanity.com/paper/2203.09645/) (2022)
- [Efficient LoFTR: Semi-Dense Local Feature Matching with Sparse-Like Speed](https://mlsanity.com/paper/2403.04765/) (2024)
- [Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentation](https://mlsanity.com/paper/2207.10866/) (2022)
- [Learning Accurate Dense Correspondences and When to Trust Them](https://mlsanity.com/paper/2101.01710/) (2021)
- [PDC-Net+: Enhanced Probabilistic Dense Correspondence Network](https://mlsanity.com/paper/2109.13912/) (2021)
- [Local All-Pair Correspondence for Point Tracking](https://mlsanity.com/paper/2407.15420/) (2024)
- [CATs: Cost Aggregation Transformers for Visual Correspondence](https://mlsanity.com/paper/2106.02520/) (2021)
- [MeshLoc: Mesh-Based Visual Localization](https://mlsanity.com/paper/2207.10762/) (2022)
- [CATs++: Boosting Cost Aggregation with Convolutions and Transformers](https://mlsanity.com/paper/2202.06817/) (2022)
- [Convolutional Hough Matching Networks](https://mlsanity.com/paper/2103.16831/) (2021)
- [Warp Consistency for Unsupervised Learning of Dense Correspondences](https://mlsanity.com/paper/2104.03308/) (2021)
- [MatchAnything: Universal Cross-Modality Image Matching with Large-Scale Pre-Training](https://mlsanity.com/paper/2501.07556/) (2025)
- [TopicFM: Robust and Interpretable Topic-Assisted Feature Matching](https://mlsanity.com/paper/2207.00328/) (2022)
- [Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondence](https://mlsanity.com/paper/2210.02689/) (2022)
- [Input-level Inductive Biases for 3D Reconstruction](https://mlsanity.com/paper/2112.03243/) (2021)
- [Deep Matching Prior: Test-Time Optimization for Dense Correspondence](https://mlsanity.com/paper/2106.03090/) (2021)
- [Adaptive Assignment for Geometry Aware Local Feature Matching](https://mlsanity.com/paper/2207.08427/) (2022)
- [TransforMatcher: Match-to-Match Attention for Semantic Correspondence](https://mlsanity.com/paper/2205.11634/) (2022)
- [PMatch: Paired Masked Image Modeling for Dense Geometric Matching](https://mlsanity.com/paper/2303.17342/) (2023)
- [Decoupling Makes Weakly Supervised Local Feature Better](https://mlsanity.com/paper/2201.02861/) (2022)
- [SFD2: Semantic-guided Feature Detection and Description](https://mlsanity.com/paper/2304.14845/) (2023)
- [2D3D-MATR: 2D-3D Matching Transformer for Detection-free Registration between Images and Point Clouds](https://mlsanity.com/paper/2308.05667/) (2023)
- [Self-Supervised Equivariant Learning for Oriented Keypoint Detection](https://mlsanity.com/paper/2204.08613/) (2022)
- [Local Feature Matching Using Deep Learning: A Survey](https://mlsanity.com/paper/2401.17592/) (2024)

## Abstract (arXiv)

In this work we target the problem of estimating accurately localised correspondences between a pair of images. We adopt the recent Neighbourhood Consensus Networks that have demonstrated promising performance for difficult correspondence problems and propose modifications to overcome their main limitations: large memory consumption, large inference time and poorly localised correspondences. Our proposed modifications can reduce the memory footprint and execution time more than 10×, with equivalent results. This is achieved by sparsifying the correlation tensor containing tentative matches, and its subsequent processing with a 4D CNN using submanifold sparse convolutions. Localisation accuracy is significantly improved by processing the input images in higher resolution, which is possible due to the reduced memory footprint, and by a novel two-stage correspondence relocalisation module. The proposed Sparse-NCNet method obtains state-of-the-art results on the HPatches Sequences and InLoc visual localisation benchmarks, and competitive results in the Aachen Day-Night benchmark.