# DeepSim-Nets: Deep Similarity Networks for Stereo Image Matching

arXiv: https://arxiv.org/abs/2304.08056 · HTML: https://mlsanity.com/paper/2304.08056/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 17 Apr 2023.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [GA-Net: Guided Aggregation Net for End-to-end Stereo Matching](https://mlsanity.com/paper/1904.06587/) (2019)
- [Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches](https://mlsanity.com/paper/1510.05970/) (2015)
- [Attentional Feature Fusion](https://mlsanity.com/paper/2009.14082/) (2020)
- [DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch](https://mlsanity.com/paper/1909.05845/) (2019)
- [Learning to Compare Image Patches via Convolutional Neural Networks](https://mlsanity.com/paper/1504.03641/) (2015)
- [Semantic Stereo for Incidental Satellite Images](https://mlsanity.com/paper/1811.08739/) (2018)
- [Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentation](https://mlsanity.com/paper/2204.07548/) (2022)
- [DublinCity: Annotated LiDAR Point Cloud and its Applications](https://mlsanity.com/paper/1909.03613/) (2019)
- [Learned Multi-Patch Similarity](https://mlsanity.com/paper/1703.08836/) (2017)
- [A New Stereo Benchmarking Dataset for Satellite Images](https://mlsanity.com/paper/1907.04404/) (2019)
- [CBMV: A Coalesced Bidirectional Matching Volume for Disparity Estimation](https://mlsanity.com/paper/1804.01967/) (2018)

## Built on by (most cited first, 3 of 3)

- [BRDF-NeRF: Neural Radiance Fields with Optical Satellite Images and BRDF Modelling](https://mlsanity.com/paper/2409.12014/) (2024)
- [Robust Confidence Intervals in Stereo Matching using Possibility Theory](https://mlsanity.com/paper/2404.06273/) (2024)
- [Multi-view dense image matching with similarity learning and geometry priors](https://mlsanity.com/paper/2505.11264/) (2025)

## Abstract (arXiv)

We present three multi-scale similarity learning architectures, or DeepSim networks. These models learn pixel-level matching with a contrastive loss and are agnostic to the geometry of the considered scene. We establish a middle ground between hybrid and end-to-end approaches by learning to densely allocate all corresponding pixels of an epipolar pair at once. Our features are learnt on large image tiles to be expressive and capture the scene's wider context. We also demonstrate that curated sample mining can enhance the overall robustness of the predicted similarities and improve the performance on radiometrically homogeneous areas. We run experiments on aerial and satellite datasets. Our DeepSim-Nets outperform the baseline hybrid approaches and generalize better to unseen scene geometries than end-to-end methods. Our flexible architecture can be readily adopted in standard multi-resolution image matching pipelines.