# ReDFeat: Recoupling Detection and Description for Multimodal Feature Learning

arXiv: https://arxiv.org/abs/2205.07439 · HTML: https://mlsanity.com/paper/2205.07439/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 49 references that are in the map.
- First posted on arXiv 16 May 2022.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [Kornia: an Open Source Differentiable Computer Vision Library for PyTorch](https://mlsanity.com/paper/1910.02190/) (2019)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [Working hard to know your neighbor's margins: Local descriptor learning loss](https://mlsanity.com/paper/1705.10872/) (2017)
- [MAGSAC++, a fast, reliable and accurate robust estimator](https://mlsanity.com/paper/1912.05909/) (2019)
- [SOSNet: Second Order Similarity Regularization for Local Descriptor Learning](https://mlsanity.com/paper/1904.05019/) (2019)
- [Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters](https://mlsanity.com/paper/1904.00889/) (2019)
- [Reinforced Feature Points: Optimizing Feature Detection and Description for a High-Level Task](https://mlsanity.com/paper/1912.00623/) (2019)
- [Fast ORB-SLAM without Keypoint Descriptors](https://mlsanity.com/paper/2008.09870/) (2020)
- [SDGMNet: Statistic-based Dynamic Gradient Modulation for Local Descriptor Learning](https://mlsanity.com/paper/2106.04434/) (2021)
- [RIFT: Multi-modal Image Matching Based on Radiation-invariant Feature Transform](https://mlsanity.com/paper/1804.09493/) (2018)

## Built on by (most cited first, 26 of 26)

- [XoFTR: Cross-modal Feature Matching Transformer](https://mlsanity.com/paper/2404.09692/) (2024)
- [Local Feature Matching Using Deep Learning: A Survey](https://mlsanity.com/paper/2401.17592/) (2024)
- [Searching from Area to Point: A Hierarchical Framework for Semantic-Geometric Combined Feature Matching](https://mlsanity.com/paper/2305.00194/) (2023)
- [Multi-Resolution SAR and Optical Remote Sensing Image Registration Methods: A Review, Datasets, and Future Perspectives](https://mlsanity.com/paper/2502.01002/) (2025)
- [R2FD2: Fast and Robust Matching of Multimodal Remote Sensing Image via Repeatable Feature Detector and Rotation-invariant Feature Descriptor](https://mlsanity.com/paper/2212.02277/) (2022)
- [3MOS: Multi-sources, Multi-resolutions, and Multi-scenes dataset for Optical-SAR image matching](https://mlsanity.com/paper/2404.00838/) (2024)
- [VideoFusion: A Spatio-Temporal Collaborative Network for Multi-modal Video Fusion](https://mlsanity.com/paper/2503.23359/) (2025)
- [M-SpecGene: Generalized Foundation Model for RGBT Multispectral Vision](https://mlsanity.com/paper/2507.16318/) (2025)
- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)
- [MIFNet: Learning Modality-Invariant Features for Generalizable Multimodal Image Matching](https://mlsanity.com/paper/2501.11299/) (2025)
- [Learning Dense Feature Matching via Lifting Single 2D Image to 3D Space](https://mlsanity.com/paper/2507.00392/) (2025)
- [DistillMatch: Leveraging Knowledge Distillation from Vision Foundation Model for Multimodal Image Matching](https://mlsanity.com/paper/2509.16017/) (2025)
- [SOMA-1M: A Large-Scale SAR-Optical Multi-resolution Alignment Dataset for Multi-Task Remote Sensing](https://mlsanity.com/paper/2602.05480/) (2026)
- [CRFT: Consistent-Recurrent Feature Flow Transformer for Cross-Modal Image Registration](https://mlsanity.com/paper/2604.05689/) (2026)
- [XPoint: A Self-Supervised Visual-State-Space based Architecture for Multispectral Image Registration](https://mlsanity.com/paper/2411.07430/) (2024)
- [OSDM-MReg: Multimodal Image Registration based One Step Diffusion Model](https://mlsanity.com/paper/2504.06027/) (2025)
- [PromptMID: Modal Invariant Descriptors Based on Diffusion and Vision Foundation Models for Optical-SAR Image Matching](https://mlsanity.com/paper/2502.18104/) (2025)
- [Multimodal Image Matching based on Frequency-domain Information of Local Energy Response](https://mlsanity.com/paper/2503.20827/) (2025)
- [AnyMatch: Supercharging Universal Multi-Modal Image Matching with Large-Scale Single-View Images](https://mlsanity.com/paper/2606.31077/) (2026)
- [REMM:Rotation-Equivariant Framework for End-to-End Multimodal Image Matching](https://mlsanity.com/paper/2407.11637/) (2024)
- [Learning Cross-Spectral Point Features with Task-Oriented Training](https://mlsanity.com/paper/2505.12593/) (2025)
- [Collaborative Learning for Unsupervised Multimodal Remote Sensing Image Registration: Integrating Self-Supervision and MIM-Guided Diffusion-Based Image Translation](https://mlsanity.com/paper/2505.22000/) (2025)
- [No Calibration, No Depth, No Problem: Cross-Sensor View Synthesis with 3D Consistency](https://mlsanity.com/paper/2602.23559/) (2026)
- [Cross-Modality Feature Fusion Based on Structured State Space Duality for Multimodal Image Registration Network](https://mlsanity.com/paper/2606.03341/) (2026)
- [Label-Free Target-Domain Adaptation for Unconstrained Event-Image Feature Matching via Dual-Stage Distillation](https://mlsanity.com/paper/2607.10082/) (2026)
- [SGFormer: Structure-Guided Transformer for Robust Local Feature Matching](https://mlsanity.com/paper/2608.03423/) (2026)

## Abstract (arXiv)

Deep-learning-based local feature extraction algorithms that combine detection and description have made significant progress in visible image matching. However, the end-to-end training of such frameworks is notoriously unstable due to the lack of strong supervision of detection and the inappropriate coupling between detection and description. The problem is magnified in cross-modal scenarios, in which most methods heavily rely on the pre-training. In this paper, we recouple independent constraints of detection and description of multimodal feature learning with a mutual weighting strategy, in which the detected probabilities of robust features are forced to peak and repeat, while features with high detection scores are emphasized during optimization. Different from previous works, those weights are detached from back propagation so that the detected probability of indistinct features would not be directly suppressed and the training would be more stable. Moreover, we propose the Super Detector, a detector that possesses a large receptive field and is equipped with learnable non-maximum suppression layers, to fulfill the harsh terms of detection. Finally, we build a benchmark that contains cross visible, infrared, near-infrared and synthetic aperture radar image pairs for evaluating the performance of features in feature matching and image registration tasks. Extensive experiments demonstrate that features trained with the recoulped detection and description, named ReDFeat, surpass previous state-of-the-arts in the benchmark, while the model can be readily trained from scratch.