# TS-Net: Combining modality specific and common features for multimodal patch matching

arXiv: https://arxiv.org/abs/1806.01550 · HTML: https://mlsanity.com/paper/1806.01550/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2018.

## Built on by (most cited first, 6 of 6)

- [Learning multi-domain feature relation for visible and Long-wave Infrared image patch matching](https://mlsanity.com/paper/2308.04880/) (2023)
- [Can representation learning for multimodal image registration be improved by supervision of intermediate layers?](https://mlsanity.com/paper/2303.00403/) (2023)
- [Relational Representation Learning Network for Cross-Spectral Image Patch Matching](https://mlsanity.com/paper/2403.11751/) (2024)
- [Why and How: Knowledge-Guided Learning for Cross-Spectral Image Patch Matching](https://mlsanity.com/paper/2412.11161/) (2024)
- [Attention-Based Multimodal Image Matching](https://mlsanity.com/paper/2103.11247/) (2021)
- [Multi-Sensor Matching with HyperNetworks](https://mlsanity.com/paper/2601.12325/) (2026)

## Abstract (arXiv)

Multimodal patch matching addresses the problem of finding the correspondences between image patches from two different modalities, e.g. RGB vs sketch or RGB vs near-infrared. The comparison of patches of different modalities can be done by discovering the information common to both modalities (Siamese like approaches) or the modality-specific information (Pseudo-Siamese like approaches). We observed that none of these two scenarios is optimal. This motivates us to propose a three-stream architecture, dubbed as TS-Net, combining the benefits of the two. In addition, we show that adding extra constraints in the intermediate layers of such networks further boosts the performance. Experimentations on three multimodal datasets show significant performance gains in comparison with Siamese and Pseudo-Siamese networks.