# Why and How: Knowledge-Guided Learning for Cross-Spectral Image Patch Matching

arXiv: https://arxiv.org/abs/2412.11161 · HTML: https://mlsanity.com/paper/2412.11161/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 56 references that are in the map.
- First posted on arXiv 15 Dec 2024.

## Builds on (most cited first, 16 of 16)

- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [HPatches: A benchmark and evaluation of handcrafted and learned local descriptors](https://mlsanity.com/paper/1704.05939/) (2017)
- [Working hard to know your neighbor's margins: Local descriptor learning loss](https://mlsanity.com/paper/1705.10872/) (2017)
- [SOSNet: Second Order Similarity Regularization for Local Descriptor Learning](https://mlsanity.com/paper/1904.05019/) (2019)
- [Learning to Compare Image Patches via Convolutional Neural Networks](https://mlsanity.com/paper/1504.03641/) (2015)
- [Beyond Cross-view Image Retrieval: Highly Accurate Vehicle Localization Using Satellite Image](https://mlsanity.com/paper/2204.04752/) (2022)
- [Hardness-Aware Deep Metric Learning](https://mlsanity.com/paper/1903.05503/) (2019)
- [Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networks](https://mlsanity.com/paper/1911.09737/) (2019)
- [Learning Spread-out Local Feature Descriptors](https://mlsanity.com/paper/1708.06320/) (2017)
- [Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimising Global Loss Functions](https://mlsanity.com/paper/1512.09272/) (2015)
- [Zero-Shot Everything Sketch-Based Image Retrieval, and in Explainable Style](https://mlsanity.com/paper/2303.14348/) (2023)
- [PN-Net: Conjoined Triple Deep Network for Learning Local Image Descriptors](https://mlsanity.com/paper/1601.05030/) (2016)
- [How to Train Triplet Networks with 100K Identities?](https://mlsanity.com/paper/1709.02940/) (2017)
- [Deep Multi-Spectral Registration Using Invariant Descriptor Learning](https://mlsanity.com/paper/1801.05171/) (2018)
- [TS-Net: Combining modality specific and common features for multimodal patch matching](https://mlsanity.com/paper/1806.01550/) (2018)

## Built on by (most cited first, 2 of 2)

- [Grid-Reg: Detector-Free Gridized Feature Learning and Matching for Large-Scale SAR-Optical Image Registration](https://mlsanity.com/paper/2507.04233/) (2025)
- [Multi-Sensor Matching with HyperNetworks](https://mlsanity.com/paper/2601.12325/) (2026)

## Abstract (arXiv)

Recently, cross-spectral image patch matching based on feature relation learning has attracted extensive attention. However, performance bottleneck problems have gradually emerged in existing methods. To address this challenge, we make the first attempt to explore a stable and efficient bridge between descriptor learning and metric learning, and construct a knowledge-guided learning network (KGL-Net), which achieves amazing performance improvements while abandoning complex network structures. Specifically, we find that there is feature extraction consistency between metric learning based on feature difference learning and descriptor learning based on Euclidean distance. This provides the foundation for bridge building. To ensure the stability and efficiency of the constructed bridge, on the one hand, we conduct an in-depth exploration of 20 combined network architectures. On the other hand, a feature-guided loss is constructed to achieve mutual guidance of features. In addition, unlike existing methods, we consider that the feature mapping ability of the metric branch should receive more attention. Therefore, a hard negative sample mining for metric learning (HNSM-M) strategy is constructed. To the best of our knowledge, this is the first time that hard negative sample mining for metric networks has been implemented and brings significant performance gains. Extensive experimental results show that our KGL-Net achieves SOTA performance in three different cross-spectral image patch matching scenarios. Our code are available at https://github.com/YuChuang1205/KGL-Net.