mlsanityData to 7 Oct 2026

XoFTR: Cross-modal Feature Matching Transformer

arXiv 2404.09692 ↗First posted 15 Apr 2024cs.CVcited by 45

Not on the map. Why?

Abstract (arXiv)

We introduce, XoFTR, a cross-modal cross-view method for local feature matching between thermal infrared (TIR) and visible images. Unlike visible images, TIR images are less susceptible to adverse lighting and weather conditions but present difficulties in matching due to significant texture and intensity differences. Current hand-crafted and learning-based methods for visible-TIR matching fall short in handling viewpoint, scale, and texture diversities. To address this, XoFTR incorporates masked image modeling pre-training and fine-tuning with pseudo-thermal image augmentation to handle the modality differences. Additionally, we introduce a refined matching pipeline that adjusts for scale discrepancies and enhances match reliability through sub-pixel level refinement. To validate our approach, we collect a comprehensive visible-thermal dataset, and show that our method outperforms existing methods on many benchmarks.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 47 of its 92 references that are in the map. First posted on arXiv 15 Apr 2024.

Builds on

47 of 92 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  7. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  8. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  9. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  10. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
  11. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  12. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
35 more
  1. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  2. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  3. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  4. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  5. 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 637
  6. 2023LightGlue: Local Feature Matching at Light Speedcited by 583
  7. 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 556
  8. 2019D2-Net: A Trainable CNN for Joint Detection and Description of Local Featurescited by 324
  9. 2016LIFT: Learned Invariant Feature Transformcited by 305
  10. 2017HPatches: A benchmark and evaluation of handcrafted and learned local descriptorscited by 261
  11. 2019DRIT++: Diverse Image-to-Image Translation via Disentangled Representationscited by 244
  12. 2021CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentationcited by 227
  13. 2022DKM: Dense Kernelized Feature Matching for Geometry Estimationcited by 174
  14. 2018Neighbourhood Consensus Networkscited by 173
  15. 2022ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformercited by 170
  16. 2017Geometry-Aware Learning of Maps for Camera Localizationcited by 161
  17. 2019GLU-Net: Global-Local Universal Network for Dense Flow and Correspondencescited by 139
  18. 2020Patch2Pix: Epipolar-Guided Pixel-Level Correspondencescited by 128
  19. 2020Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translationcited by 66
  20. 2021Dual Contrastive Learning for Unsupervised Image-to-Image Translationcited by 65
  21. 2023FasterViT: Fast Vision Transformers with Hierarchical Attentioncited by 65
  22. 2020GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Networkcited by 53
  23. 2019Borrow from Anywhere: Pseudo Multi-modal Object Detection in Thermal Imagerycited by 38
  24. 2022TopicFM: Robust and Interpretable Topic-Assisted Feature Matchingcited by 32
  25. 2022Adaptive Assignment for Geometry Aware Local Feature Matchingcited by 30
  26. 2023Edge-guided Multi-domain RGB-to-TIR image Translation for Training Vision Tasks with Challenging Labelscited by 30
  27. 2023PMatch: Paired Masked Image Modeling for Dense Geometric Matchingcited by 29
  28. 2023Complementary Random Masking for RGB-Thermal Semantic Segmentationcited by 27
  29. 2018Synthetic data generation for end-to-end thermal infrared trackingcited by 26
  30. 2022ReDFeat: Recoupling Detection and Description for Multimodal Feature Learningcited by 26
  31. 2023DeepMatcher: A Deep Transformer-based Network for Robust and Accurate Local Feature Matchingcited by 18
  32. 2018Joint detection and matching of feature points in multimodal imagescited by 10
  33. 2023TopicFM+: Boosting Accuracy and Efficiency of Topic-Assisted Feature Matchingcited by 10
  34. 2022RGB-Multispectral Matching: Dataset, Learning Methodology, Evaluationcited by 6
  35. 2023Incomplete Multimodal Learning for Remote Sensing Data Fusioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 16; 2026: 27.

45 papers.

Sort:

By area:

  1. 2025MatchAnything: Universal Cross-Modality Image Matching with Large-Scale Pre-Trainingcited by 40
  2. 2025JamMa: Ultra-lightweight Local Feature Matching with Joint Mambacited by 14
  3. 2025ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translationcited by 13
  4. 2025OrthoLoC: UAV 6-DoF Localization and Calibration Using Orthographic Geodatacited by 10
  5. 2025Multi-Resolution SAR and Optical Remote Sensing Image Registration Methods: A Review, Datasets, and Future Perspectivescited by 9
  6. 2025Deep Learning Reforms Image Matching: A Survey and Outlookcited by 7
  7. 2025MIFNet: Learning Modality-Invariant Features for Generalizable Multimodal Image Matchingcited by 6
  8. 2025MapGlue: Multimodal Remote Sensing Image Matchingcited by 5
  9. 2025Learning Dense Feature Matching via Lifting Single 2D Image to 3D Spacecited by 4
  10. 2025GDROS: A Geometry-Guided Dense Registration Framework for Optical-SAR Images under Large Geometric Transformationscited by 4
  11. 2025A 3D Cross-modal Keypoint Descriptor for MR-US Matching and Registrationcited by 3
  12. 2026SOMA-1M: A Large-Scale SAR-Optical Multi-resolution Alignment Dataset for Multi-Task Remote Sensingcited by 3
33 more
  1. 2026TherA: Thermal-Aware Visual-Language Prompting for Controllable RGB-to-Thermal Infrared Translationcited by 3
  2. 2026CRFT: Consistent-Recurrent Feature Flow Transformer for Cross-Modal Image Registrationcited by 3
  3. 2024XPoint: A Self-Supervised Visual-State-Space based Architecture for Multispectral Image Registrationcited by 2
  4. 2026Are Pretrained Image Matchers Good Enough for SAR-Optical Satellite Registration?cited by 2
  5. 2025PromptMID: Modal Invariant Descriptors Based on Diffusion and Vision Foundation Models for Optical-SAR Image Matchingcited by 1
  6. 2026CM-Bench: A Comprehensive Cross-Modal Feature Matching Benchmark Bridging Visible and Infrared Imagescited by 1
  7. 2026SEAR: Simple and Efficient Adaptation of Visual Geometric Transformers for Unpaired RGB+Thermal 3D Reconstructioncited by 1
  8. 2024REMM:Rotation-Equivariant Framework for End-to-End Multimodal Image Matchingcited by 0
  9. 2025Flow Intelligence: Robust Feature Matching via Temporal Signature Correlationcited by 0
  10. 2025Grid-Reg: Detector-Free Gridized Feature Learning and Matching for Large-Scale SAR-Optical Image Registrationcited by 0
  11. 2025Semi-distributed Cross-modal Air-Ground Relative Localizationcited by 0
  12. 2025Coarse-to-Fine Non-Rigid Registration for Side-Scan Sonar Mosaickingcited by 0
  13. 2026OpenNavMap: Multi-Session Appearance-Based Topometric Mapping for Scalable Visual Navigationcited by 0
  14. 2026Multi-Sensor Matching with HyperNetworkscited by 0
  15. 2026LST-SLAM: A Stereo Thermal SLAM System for Kilometer-Scale Dynamic Environmentscited by 0
  16. 2026No Calibration, No Depth, No Problem: Cross-Sensor View Synthesis with 3D Consistencycited by 0
  17. 2026SCC-Loc: A Unified Semantic Cascade Consensus Framework for UAV Thermal Geo-Localizationcited by 0
  18. 2026Particle Diffusion Matching: Random Walk Correspondence Search for the Alignment of Standard and Ultra-Widefield Fundus Imagescited by 0
  19. 2026The Spectrascapes Dataset: Street-view imagery beyond the visible captured using a mobile platformcited by 0
  20. 2026Preserve, Then Resolve: Many-to-Many Association and Robust Estimation with General-Purpose Visual Featurescited by 0
  21. 2026TAR: Text Semantic Assisted Cross-modal Image Registration Framework for Optical and SAR Imagescited by 0
  22. 2026Cross-Modality Feature Fusion Based on Structured State Space Duality for Multimodal Image Registration Networkcited by 0
  23. 2026Unpaired RGB-Thermal Gaussian-Splatting Using Visual Geometric Transformerscited by 0
  24. 2026RBE-Flow: Recurrent Bayesian Estimation on Feature Manifolds for Cross-Modal Registrationcited by 0
  25. 2026AnyMatch: Supercharging Universal Multi-Modal Image Matching with Large-Scale Single-View Imagescited by 0
  26. 2026Label-Free Target-Domain Adaptation for Unconstrained Event-Image Feature Matching via Dual-Stage Distillationcited by 0
  27. 2026Image-to-Point Cloud Registration Made Easy with Rectified Flow-based LiDAR Upsamplingcited by 0
  28. 2026Robust RPC Bundle Adjustment for Multi-Date Satellite Imagery with Season-Invariant Correspondencescited by 0
  29. 2026SGFormer: Structure-Guided Transformer for Robust Local Feature Matchingcited by 0
  30. 2026Robust structure from motion for aerial-ground images via detector-free feature matching and multi-view track refinementcited by 0
  31. 2026Cross-Spectral Dense Correspondence for Multimodal Spectral Medical Imagingcited by 0
  32. 2026CrossFeat: Bridging Imaging Modalities in Feature Descriptor Spacecited by 0
  33. 2026Radiation, Rotation and Scale Invariant Feature Descriptor for Multimodal Image Matchingcited by 0