# TopicFM: Robust and Interpretable Topic-Assisted Feature Matching

arXiv: https://arxiv.org/abs/2207.00328 · HTML: https://mlsanity.com/paper/2207.00328/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 76 references that are in the map.
- First posted on arXiv 1 Jul 2022.

## Builds on (most cited first, 30 of 46)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Transformer Interpretability Beyond Attention Visualization](https://mlsanity.com/paper/2012.09838/) (2020)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [Efficient Attention: Attention with Linear Complexities](https://mlsanity.com/paper/1812.01243/) (2018)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [HPatches: A benchmark and evaluation of handcrafted and learned local descriptors](https://mlsanity.com/paper/1704.05939/) (2017)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [COTR: Correspondence Transformer for Matching Across Images](https://mlsanity.com/paper/2103.14167/) (2021)
- [Learning to Find Good Correspondences](https://mlsanity.com/paper/1711.05971/) (2017)
- [Neighbourhood Consensus Networks](https://mlsanity.com/paper/1810.10510/) (2018)
- [Interpretable Convolutional Neural Networks](https://mlsanity.com/paper/1710.00935/) (2017)
- [LF-Net: Learning Local Features from Images](https://mlsanity.com/paper/1805.09662/) (2018)

## Built on by (most cited first, 30 of 32)

- [Efficient LoFTR: Semi-Dense Local Feature Matching with Sparse-Like Speed](https://mlsanity.com/paper/2403.04765/) (2024)
- [XoFTR: Cross-modal Feature Matching Transformer](https://mlsanity.com/paper/2404.09692/) (2024)
- [MESA: Matching Everything by Segmenting Anything](https://mlsanity.com/paper/2401.16741/) (2024)
- [CoMatch: Dynamic Covisibility-Aware Transformer for Bilateral Subpixel-Level Semi-Dense Image Matching](https://mlsanity.com/paper/2503.23925/) (2025)
- [TopicFM+: Boosting Accuracy and Efficiency of Topic-Assisted Feature Matching](https://mlsanity.com/paper/2307.00485/) (2023)
- [Learning to Produce Semi-dense Correspondences for Visual Localization](https://mlsanity.com/paper/2402.08359/) (2024)
- [RUBIK: A Structured Benchmark for Image Matching across Geometric Challenges](https://mlsanity.com/paper/2502.19955/) (2025)
- [OAMatcher: An Overlapping Areas-based Network for Accurate Local Feature Matching](https://mlsanity.com/paper/2302.05846/) (2023)
- [EDM: Efficient Deep Feature Matching](https://mlsanity.com/paper/2503.05122/) (2025)
- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)
- [Affine steerers for structured keypoint description](https://mlsanity.com/paper/2408.14186/) (2024)
- [MESA: Effective Matching Redundancy Reduction by Semantic Area Segmentation](https://mlsanity.com/paper/2408.00279/) (2024)
- [FMRT: Learning Accurate Feature Matching with Reconciliatory Transformer](https://mlsanity.com/paper/2310.13605/) (2023)
- [A Robust Multisource Remote Sensing Image Matching Method Utilizing Attention and Feature Enhancement Against Noise Interference](https://mlsanity.com/paper/2410.11848/) (2024)
- [UFM: Unified Feature Matching Pre-training with Multi-Modal Image Assistants](https://mlsanity.com/paper/2503.21820/) (2025)
- [CasP: Improving Semi-Dense Feature Matching Pipeline Leveraging Cascaded Correspondence Priors for Guidance](https://mlsanity.com/paper/2507.17312/) (2025)
- [SGAD: Semantic and Geometric-aware Descriptor for Local Feature Matching](https://mlsanity.com/paper/2508.02278/) (2025)
- [SURE: Semi-dense Uncertainty-REfined Feature Matching](https://mlsanity.com/paper/2603.04869/) (2026)
- [CM-Bench: A Comprehensive Cross-Modal Feature Matching Benchmark Bridging Visible and Infrared Images](https://mlsanity.com/paper/2603.12690/) (2026)
- [Revisiting Change Detection Methods for their Application to Serac Fall Time-Lapse Monitoring](https://mlsanity.com/paper/2605.28100/) (2026)
- [Improved 3D Point-Line Mapping Regression for Camera Relocalization](https://mlsanity.com/paper/2502.20814/) (2025)
- [CQVPR: Landmark-aware Contextual Queries for Visual Place Recognition](https://mlsanity.com/paper/2503.08170/) (2025)
- [Good Keypoints for the Two-View Geometry Estimation Problem](https://mlsanity.com/paper/2503.18767/) (2025)
- [Not All Pixels Are Equal: Confidence-Guided Attention for Feature Matching](https://mlsanity.com/paper/2505.02161/) (2025)
- [Semantic and Feature Guided Uncertainty Quantification of Visual Localization for Autonomous Vehicles](https://mlsanity.com/paper/2506.15851/) (2025)
- [Coarse-to-Fine Non-Rigid Registration for Side-Scan Sonar Mosaicking](https://mlsanity.com/paper/2512.00052/) (2025)
- [Improving Local Feature Matching by Entropy-inspired Scale Adaptability and Flow-endowed Local Consistency](https://mlsanity.com/paper/2604.06713/) (2026)
- [REDI-Match: Rotation-Equivariant Distillation for Efficient and Robust Dense Matching](https://mlsanity.com/paper/2606.24330/) (2026)
- [RBE-Flow: Recurrent Bayesian Estimation on Feature Manifolds for Cross-Modal Registration](https://mlsanity.com/paper/2606.30492/) (2026)
- [SGFormer: Structure-Guided Transformer for Robust Local Feature Matching](https://mlsanity.com/paper/2608.03423/) (2026)

## Abstract (arXiv)

This study addresses an image-matching problem in challenging cases, such as large scene variations or textureless scenes. To gain robustness to such situations, most previous studies have attempted to encode the global contexts of a scene via graph neural networks or transformers. However, these contexts do not explicitly represent high-level contextual information, such as structural shapes or semantic instances; therefore, the encoded features are still not sufficiently discriminative in challenging scenes. We propose a novel image-matching method that applies a topic-modeling strategy to encode high-level contexts in images. The proposed method trains latent semantic instances called topics. It explicitly models an image as a multinomial distribution of topics, and then performs probabilistic feature matching. This approach improves the robustness of matching by focusing on the same semantic areas between the images. In addition, the inferred topics provide interpretability for matching the results, making our method explainable. Extensive experiments on outdoor and indoor datasets show that our method outperforms other state-of-the-art methods, particularly in challenging cases. The code is available at https://github.com/TruongKhang/TopicFM.