# Spatially Optimized Compact Deep Metric Learning Model for Similarity Search

arXiv: https://arxiv.org/abs/2404.06593 · HTML: https://mlsanity.com/paper/2404.06593/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 9 Apr 2024.

## Builds on (most cited first, 12 of 12)

- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning](https://mlsanity.com/paper/1904.06627/) (2019)
- [Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2109.14545/) (2021)
- [A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses](https://mlsanity.com/paper/2003.08983/) (2020)
- [Involution: Inverting the Inherence of Convolution for Visual Recognition](https://mlsanity.com/paper/2103.06255/) (2021)
- [Deep Network Approximation: Beyond ReLU to Diverse Activation Functions](https://mlsanity.com/paper/2307.06555/) (2023)
- [GELU Activation Function in Deep Learning: A Comprehensive Mathematical Analysis and Performance](https://mlsanity.com/paper/2305.12073/) (2023)
- [Deep Metric Learning-based Image Retrieval System for Chest Radiograph and its Clinical Applications in COVID-19](https://mlsanity.com/paper/2012.03663/) (2020)
- [Metric Learning for Dynamic Text Classification](https://mlsanity.com/paper/1911.01026/) (2019)
- [An Analysis of State-of-the-art Activation Functions For Supervised Deep Neural Network](https://mlsanity.com/paper/2104.02523/) (2021)
- [Automatic Modulation Classification Using Involution Enabled Residual Networks](https://mlsanity.com/paper/2108.10001/) (2021)
- [Involution Fused ConvNet for Classifying Eye-Tracking Patterns of Children with Autism Spectrum Disorder](https://mlsanity.com/paper/2401.03575/) (2024)

## Abstract (arXiv)

Spatial optimization is often overlooked in many computer vision tasks. Filters should be able to recognize the features of an object regardless of where it is in the image. Similarity search is a crucial task where spatial features decide an important output. The capacity of convolution to capture visual patterns across various locations is limited. In contrast to convolution, the involution kernel is dynamically created at each pixel based on the pixel value and parameters that have been learned. This study demonstrates that utilizing a single layer of involution feature extractor alongside a compact convolution model significantly enhances the performance of similarity search. Additionally, we improve predictions by using the GELU activation function rather than the ReLU. The negligible amount of weight parameters in involution with a compact model with better performance makes the model very useful in real-world implementations. Our proposed model is below 1 megabyte in size. We have experimented with our proposed methodology and other models on CIFAR-10, FashionMNIST, and MNIST datasets. Our proposed method outperforms across all three datasets.