# Learning an Adaptation Function to Assess Image Visual Similarities

arXiv: https://arxiv.org/abs/2206.01417 · HTML: https://mlsanity.com/paper/2206.01417/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 3 Jun 2022.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [Sampling Matters in Deep Embedding Learning](https://mlsanity.com/paper/1706.07567/) (2017)
- [Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future](https://mlsanity.com/paper/2001.07092/) (2020)
- [Improving Generalization via Scalable Neighborhood Component Analysis](https://mlsanity.com/paper/1808.04699/) (2018)
- [Revealing interpretable object representations from human behavior](https://mlsanity.com/paper/1901.02915/) (2019)
- [Totally Looks Like - How Humans Compare, Compared to Machines](https://mlsanity.com/paper/1803.01485/) (2018)
- [Finding your Lookalike: Measuring Face Similarity Rather than Face Identity](https://mlsanity.com/paper/1806.05252/) (2018)
- [MosAIc: Finding Artistic Connections across Culture with Conditional Image Retrieval](https://mlsanity.com/paper/2007.07177/) (2020)
- [High-Level Perceptual Similarity is Enabled by Learning Diverse Tasks](https://mlsanity.com/paper/1903.10920/) (2019)

## Built on by (most cited first, 1 of 1)

- [Fine-tune your Classifier: Finding Correlations With Temperature](https://mlsanity.com/paper/2210.09715/) (2022)

## Abstract (arXiv)

Human perception is routinely assessing the similarity between images, both for decision making and creative thinking. But the underlying cognitive process is not really well understood yet, hence difficult to be mimicked by computer vision systems. State-of-the-art approaches using deep architectures are often based on the comparison of images described as feature vectors learned for image categorization task. As a consequence, such features are powerful to compare semantically related images but not really efficient to compare images visually similar but semantically unrelated. Inspired by previous works on neural features adaptation to psycho-cognitive representations, we focus here on the specific task of learning visual image similarities when analogy matters. We propose to compare different supervised, semi-supervised and self-supervised networks, pre-trained on distinct scales and contents datasets (such as ImageNet-21k, ImageNet-1K or VGGFace2) to conclude which model may be the best to approximate the visual cortex and learn only an adaptation function corresponding to the approximation of the the primate IT cortex through the metric learning framework. Our experiments conducted on the Totally Looks Like image dataset highlight the interest of our method, by increasing the retrieval scores of the best model @1 by 2.25x. This research work was recently accepted for publication at the ICIP 2021 international conference [1]. In this new article, we expand on this previous work by using and comparing new pre-trained feature extractors on other datasets.