mlsanityData to 14 Oct 2026

UFM: Unified Feature Matching Pre-training with Multi-Modal Image Assistants

arXiv 2503.21820 ↗First posted 26 Mar 2025cs.CV eess.IVcited by 1

Not on the map. Why?

Abstract (arXiv)

Image feature matching, a foundational task in computer vision, remains challenging for multimodal image applications, often necessitating intricate training on specific datasets. In this paper, we introduce a Unified Feature Matching pre-trained model (UFM) designed to address feature matching challenges across a wide spectrum of modal images. We present Multimodal Image Assistant (MIA) transformers, finely tunable structures adept at handling diverse feature matching problems. UFM exhibits versatility in addressing both feature matching tasks within the same modal and those across different modals. Additionally, we propose a data augmentation algorithm and a staged pre-training strategy to effectively tackle challenges arising from sparse data in specific modals and imbalanced modal datasets. Experimental results demonstrate that UFM excels in generalization and performance across various feature matching tasks. The code will be released at:https://github.com/LiaoYun0x0/UFM.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 43 of its 88 references that are in the map. First posted on arXiv 26 Mar 2025.

Builds on

43 of 88 references.

References · 43

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  5. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  6. 2022Visual Prompt Tuningcited by 1,681
  7. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  8. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  9. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  10. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  11. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 737
  12. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 670
31 more
  1. 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 559
  2. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 425
  3. 2019D2-Net: A Trainable CNN for Joint Detection and Description of Local Featurescited by 326
  4. 2016LIFT: Learned Invariant Feature Transformcited by 306
  5. 2017HPatches: A benchmark and evaluation of handcrafted and learned local descriptorscited by 264
  6. 2015YFCC100M: The New Data in Multimedia Researchcited by 235
  7. 2017Working hard to know your neighbor's margins: Local descriptor learning losscited by 212
  8. 2021Back to the Feature: Learning Robust Camera Localization from Pixels to Posecited by 199
  9. 2021COTR: Correspondence Transformer for Matching Across Imagescited by 183
  10. 2022DKM: Dense Kernelized Feature Matching for Geometry Estimationcited by 177
  11. 2020ASLFeat: Learning Local Features of Accurate Shape and Localizationcited by 130
  12. 2020Patch2Pix: Epipolar-Guided Pixel-Level Correspondencescited by 129
  13. 2020More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classificationcited by 126
  14. 2022MatchFormer: Interleaving Attention in Transformers for Feature Matchingcited by 126
  15. 2020MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Maskcited by 117
  16. 2021Learning Accurate Dense Correspondences and When to Trust Themcited by 113
  17. 2018Urban Change Detection for Multispectral Earth Observation Using Convolutional Neural Networkscited by 110
  18. 2020Learning Feature Descriptors using Camera Pose Supervisioncited by 104
  19. 2019Hierarchical Scene Coordinate Classification and Regression for Visual Localizationcited by 93
  20. 2021PDC-Net+: Enhanced Probabilistic Dense Correspondence Networkcited by 80
  21. 2022Visual Prompt Tuning for Generative Transfer Learningcited by 75
  22. 2022Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Imagecited by 42
  23. 2022Affine Medical Image Registration with Coarse-to-Fine Vision Transformercited by 35
  24. 2022TopicFM: Robust and Interpretable Topic-Assisted Feature Matchingcited by 32
  25. 2022Adaptive Assignment for Geometry Aware Local Feature Matchingcited by 30
  26. 2015WxBS: Wide Baseline Stereo Generalizationscited by 23
  27. 2023DeepMatcher: A Deep Transformer-based Network for Robust and Accurate Local Feature Matchingcited by 18
  28. 2020Image Stylization for Robust Featurescited by 11
  29. 2022R2FD2: Fast and Robust Matching of Multimodal Remote Sensing Image via Repeatable Feature Detector and Rotation-invariant Feature Descriptorcited by 8
  30. 2023OAMatcher: An Overlapping Areas-based Network for Accurate Local Feature Matchingcited by 7
  31. 2021Retrieval and Localization with Observation Constraintscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026MV-RoMa: From Pairwise Matching into Multi-View Track Reconstructioncited by 2