# Love Thy Neighbors: Image Annotation by Exploiting Image Metadata

arXiv: https://arxiv.org/abs/1508.07647 · HTML: https://mlsanity.com/paper/1508.07647/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Aug 2015.

## Built on by (most cited first, 4 of 4)

- [Learning Visual Representations via Language-Guided Sampling](https://mlsanity.com/paper/2302.12248/) (2023)
- [Reviving the Context: Camera Trap Species Classification as Link Prediction on Multimodal Knowledge Graphs](https://mlsanity.com/paper/2401.00608/) (2023)
- [Multi-Label Classification on Remote-Sensing Images](https://mlsanity.com/paper/2201.01971/) (2022)
- [Visual Transformers with Primal Object Queries for Multi-Label Image Classification](https://mlsanity.com/paper/2112.05485/) (2021)

## Abstract (arXiv)

Some images that are difficult to recognize on their own may become more clear in the context of a neighborhood of related images with similar social-network metadata. We build on this intuition to improve multilabel image annotation. Our model uses image metadata nonparametrically to generate neighborhoods of related images using Jaccard similarities, then uses a deep neural network to blend visual information from the image and its neighbors. Prior work typically models image metadata parametrically, in contrast, our nonparametric treatment allows our model to perform well even when the vocabulary of metadata changes between training and testing. We perform comprehensive experiments on the NUS-WIDE dataset, where we show that our model outperforms state-of-the-art methods for multilabel image annotation even when our model is forced to generalize to new types of metadata.