# Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimising Global Loss Functions

arXiv: https://arxiv.org/abs/1512.09272 · HTML: https://mlsanity.com/paper/1512.09272/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2015.

## Built on by (most cited first, 30 of 33)

- [Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose](https://mlsanity.com/paper/2110.14213/) (2021)
- [Density-invariant Features for Distant Point Cloud Registration](https://mlsanity.com/paper/2307.09788/) (2023)
- [Spectral, Probabilistic, and Deep Metric Learning: Tutorial and Survey](https://mlsanity.com/paper/2201.09267/) (2022)
- [Contrastive Bayesian Analysis for Deep Metric Learning](https://mlsanity.com/paper/2210.04402/) (2022)
- [3D reconstruction from spherical images: A review of techniques, applications, and prospects](https://mlsanity.com/paper/2302.04495/) (2023)
- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)
- [Multi-level Distance Regularization for Deep Metric Learning](https://mlsanity.com/paper/2102.04223/) (2021)
- [Frequency-Semantic Enhanced Variational Autoencoder for Zero-Shot Skeleton-based Action Recognition](https://mlsanity.com/paper/2506.22179/) (2025)
- [SDGMNet: Statistic-based Dynamic Gradient Modulation for Local Descriptor Learning](https://mlsanity.com/paper/2106.04434/) (2021)
- [DeepBrainPrint: A Novel Contrastive Framework for Brain MRI Re-Identification](https://mlsanity.com/paper/2302.13057/) (2023)
- [RobustMat: Neural Diffusion for Street Landmark Patch Matching under Challenging Environments](https://mlsanity.com/paper/2311.03904/) (2023)
- [Image Patch-Matching with Graph-Based Learning in Street Scenes](https://mlsanity.com/paper/2311.04617/) (2023)
- [DynamicSurf: Dynamic Neural RGB-D Surface Reconstruction with an Optimizable Feature Grid](https://mlsanity.com/paper/2311.08159/) (2023)
- [Learning multi-domain feature relation for visible and Long-wave Infrared image patch matching](https://mlsanity.com/paper/2308.04880/) (2023)
- [Semi Supervised Deep Quick Instance Detection and Segmentation](https://mlsanity.com/paper/2101.06405/) (2021)
- [Relational Representation Learning Network for Cross-Spectral Image Patch Matching](https://mlsanity.com/paper/2403.11751/) (2024)
- [Audio-visual cross-modality knowledge transfer for machine learning-based in-situ monitoring in laser additive manufacturing](https://mlsanity.com/paper/2408.05307/) (2024)
- [Why and How: Knowledge-Guided Learning for Cross-Spectral Image Patch Matching](https://mlsanity.com/paper/2412.11161/) (2024)
- [Kinship Verification Based on Cross-Generation Feature Interaction Learning](https://mlsanity.com/paper/2109.02809/) (2021)
- [Energy-bounded Learning for Robust Models of Code](https://mlsanity.com/paper/2112.11226/) (2021)
- [Self-supervised Latent Space Optimization with Nebula Variational Coding](https://mlsanity.com/paper/2506.01414/) (2025)
- [SVMax: A Feature Embedding Regularizer](https://mlsanity.com/paper/2103.02770/) (2021)
- [Ordinal-Quadruplet: Retrieval of Missing Classes in Ordinal Time Series](https://mlsanity.com/paper/2201.09907/) (2022)
- [A Gis Aided Approach for Geolocalizing an Unmanned Aerial System Using Deep Learning](https://mlsanity.com/paper/2208.12251/) (2022)
- [Learning Deep Optimal Embeddings with Sinkhorn Divergences](https://mlsanity.com/paper/2209.06469/) (2022)
- [Multimorbidity Content-Based Medical Image Retrieval Using Proxies](https://mlsanity.com/paper/2211.12185/) (2022)
- [Self-supervised Training Sample Difficulty Balancing for Local Descriptor Learning](https://mlsanity.com/paper/2303.06124/) (2023)
- [Shadow loss: Memory-linear deep metric learning for efficient training](https://mlsanity.com/paper/2311.14012/) (2023)
- [Learn and Search: An Elegant Technique for Object Lookup using Contrastive Learning](https://mlsanity.com/paper/2403.07231/) (2024)
- [HALO: Robust Out-of-Distribution Detection via Joint Optimisation](https://mlsanity.com/paper/2502.19755/) (2025)

## Abstract (arXiv)

Recent innovations in training deep convolutional neural network (ConvNet) models have motivated the design of new methods to automatically learn local image descriptors. The latest deep ConvNets proposed for this task consist of a siamese network that is trained by penalising misclassification of pairs of local image patches. Current results from machine learning show that replacing this siamese by a triplet network can improve the classification accuracy in several problems, but this has yet to be demonstrated for local image descriptor learning. Moreover, current siamese and triplet networks have been trained with stochastic gradient descent that computes the gradient from individual pairs or triplets of local image patches, which can make them prone to overfitting. In this paper, we first propose the use of triplet networks for the problem of local image descriptor learning. Furthermore, we also propose the use of a global loss that minimises the overall classification error in the training set, which can improve the generalisation capability of the model. Using the UBC benchmark dataset for comparing local image descriptors, we show that the triplet network produces a more accurate embedding than the siamese network in terms of the UBC dataset errors. Moreover, we also demonstrate that a combination of the triplet and global losses produces the best embedding in the field, using this triplet network. Finally, we also show that the use of the central-surround siamese network trained with the global loss produces the best result of the field on the UBC dataset. Pre-trained models are available online at https://github.com/vijaykbg/deep-patchmatch