# Needle In A Haystack, Fast: Benchmarking Image Perceptual Similarity Metrics At Scale

arXiv: https://arxiv.org/abs/2206.00282 · HTML: https://mlsanity.com/paper/2206.00282/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 58 references that are in the map.
- First posted on arXiv 1 Jun 2022.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [HPatches: A benchmark and evaluation of handcrafted and learned local descriptors](https://mlsanity.com/paper/1704.05939/) (2017)
- [On the Origins of Memes by Means of Fringe Web Communities](https://mlsanity.com/paper/1805.12512/) (2018)
- [Learning to Break Deep Perceptual Hashing: The Use Case NeuralHash](https://mlsanity.com/paper/2111.06628/) (2021)
- [Image Provenance Analysis at Scale](https://mlsanity.com/paper/1801.06510/) (2018)
- [MemeSequencer: Sparse Matching for Embedding Image Macros](https://mlsanity.com/paper/1802.04936/) (2018)
- [VoterFraud2020: a Multi-modal Dataset of Election Fraud Claims on Twitter](https://mlsanity.com/paper/2101.08210/) (2021)
- [Motif Mining: Finding and Summarizing Remixed Image Content](https://mlsanity.com/paper/2203.08327/) (2022)
- [LATCH: Learned Arrangements of Three Patch Codes](https://mlsanity.com/paper/1501.03719/) (2015)
- [Provenance Filtering for Multimedia Phylogeny](https://mlsanity.com/paper/1706.00447/) (2017)
- [Revisiting Binary Local Image Description for Resource Limited Devices](https://mlsanity.com/paper/2108.08380/) (2021)
- [Dataset and Case Studies for Visual Near-Duplicates Detection in the Context of Social Media](https://mlsanity.com/paper/2203.07167/) (2022)

## Built on by (most cited first, 1 of 1)

- [A Practical Guide to Sample-based Statistical Distances for Evaluating Generative Models in Science](https://mlsanity.com/paper/2403.12636/) (2024)

## Abstract (arXiv)

The advent of the internet, followed shortly by the social media made it ubiquitous in consuming and sharing information between anyone with access to it. The evolution in the consumption of media driven by this change, led to the emergence of images as means to express oneself, convey information and convince others efficiently. With computer vision algorithms progressing radically over the last decade, it is become easier and easier to study at scale the role of images in the flow of information online. While the research questions and overall pipelines differ radically, almost all start with a crucial first step - evaluation of global perceptual similarity between different images. That initial step is crucial for overall pipeline performance and processes most images. A number of algorithms are available and currently used to perform it, but so far no comprehensive review was available to guide the choice of researchers as to the choice of an algorithm best suited to their question, assumptions and computational resources. With this paper we aim to fill this gap, showing that classical computer vision methods are not necessarily the best approach, whereas a pair of relatively little used methods - Dhash perceptual hash and SimCLR v2 ResNets achieve excellent performance, scale well and are computationally efficient.