# Multi-Sensor Matching with HyperNetworks

arXiv: https://arxiv.org/abs/2601.12325 · HTML: https://mlsanity.com/paper/2601.12325/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 41 of its 96 references that are in the map.
- First posted on arXiv 18 Jan 2026.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [A Learned Representation For Artistic Style](https://mlsanity.com/paper/1610.07629/) (2016)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [Coordinate Attention for Efficient Mobile Network Design](https://mlsanity.com/paper/2103.02907/) (2021)
- [LLVIP: A Visible-infrared Paired Dataset for Low-light Vision](https://mlsanity.com/paper/2108.10831/) (2021)
- [Working hard to know your neighbor's margins: Local descriptor learning loss](https://mlsanity.com/paper/1705.10872/) (2017)
- [Drone-based RGB-Infrared Cross-Modality Vehicle Detection via Uncertainty-Aware Learning](https://mlsanity.com/paper/2003.02437/) (2020)
- [Learning to Compare Image Patches via Convolutional Neural Networks](https://mlsanity.com/paper/1504.03641/) (2015)
- [Transformers as Meta-Learners for Implicit Neural Representations](https://mlsanity.com/paper/2208.02801/) (2022)
- [HyperSeg: Patch-wise Hypernetwork for Real-time Semantic Segmentation](https://mlsanity.com/paper/2012.11582/) (2020)
- [Deep Meta Functionals for Shape Representation](https://mlsanity.com/paper/1908.06277/) (2019)
- [XoFTR: Cross-modal Feature Matching Transformer](https://mlsanity.com/paper/2404.09692/) (2024)
- [MINIMA: Modality Invariant Image Matching](https://mlsanity.com/paper/2412.19412/) (2024)
- [Bridging Remote Sensors with Multisensor Geospatial Foundation Models](https://mlsanity.com/paper/2404.01260/) (2024)
- [Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networks](https://mlsanity.com/paper/1911.09737/) (2019)
- [MatchAnything: Universal Cross-Modality Image Matching with Large-Scale Pre-Training](https://mlsanity.com/paper/2501.07556/) (2025)
- [Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimising Global Loss Functions](https://mlsanity.com/paper/1512.09272/) (2015)
- [Modality Unifying Network for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2309.06262/) (2023)
- [PN-Net: Conjoined Triple Deep Network for Learning Local Image Descriptors](https://mlsanity.com/paper/1601.05030/) (2016)
- [DiffV2IR: Visible-to-Infrared Diffusion Model via Vision-Language Understanding](https://mlsanity.com/paper/2503.19012/) (2025)
- [Multi-sensor large-scale dataset for multi-view 3D reconstruction](https://mlsanity.com/paper/2203.06111/) (2022)
- [Reliability-Aware Prediction via Uncertainty Learning for Person Image Retrieval](https://mlsanity.com/paper/2210.13440/) (2022)

## Abstract (arXiv)

Hypernetworks are models that generate or modulate the weights of another network. They provide a flexible mechanism for injecting context and task conditioning and have proven broadly useful across diverse applications without significant increases in model size. We leverage hypernetworks to improve multimodal patch matching by introducing a lightweight descriptor-learning architecture that augments a Siamese CNN with (i) hypernetwork modules that compute adaptive, per-channel scaling and shifting and (ii) conditional instance normalization that provides modality-specific adaptation (e.g., visible vs. infrared, VIS-IR) in shallow layers. This combination preserves the efficiency of descriptor-based methods during inference while increasing robustness to appearance shifts. Trained with a triplet loss and hard-negative mining, our approach achieves state-of-the-art results on VIS-NIR and other VIS-IR benchmarks and matches or surpasses prior methods on additional datasets, despite their higher inference cost. To spur progress on domain shift, we also release GAP-VIR, a cross-platform (ground/aerial) VIS-IR patch dataset with 500K pairs, enabling rigorous evaluation of cross-domain generalization and adaptation.