# The Met Dataset: Instance-level Recognition for Artworks

arXiv: https://arxiv.org/abs/2202.01747 · HTML: https://mlsanity.com/paper/2202.01747/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 3 Feb 2022.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [Likelihood Ratios for Out-of-Distribution Detection](https://mlsanity.com/paper/1906.02845/) (2019)
- [Recent Advances in Open Set Recognition: A Survey](https://mlsanity.com/paper/1811.08581/) (2018)
- [Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval](https://mlsanity.com/paper/2004.01804/) (2020)
- [NormFace: L2 Hypersphere Embedding for Face Verification](https://mlsanity.com/paper/1704.06369/) (2017)
- [Billion-scale semi-supervised learning for image classification](https://mlsanity.com/paper/1905.00546/) (2019)
- [How to Read Paintings: Semantic Art Understanding with Multi-Modal Retrieval](https://mlsanity.com/paper/1810.09617/) (2018)
- [Recognizing Image Style](https://mlsanity.com/paper/1311.3715/) (2013)
- [Learning and aggregating deep local descriptors for instance-level recognition](https://mlsanity.com/paper/2007.13172/) (2020)
- [BAM! The Behance Artistic Media Dataset for Recognition Beyond Photography](https://mlsanity.com/paper/1704.08614/) (2017)
- [Products-10K: A Large-scale Product Recognition Dataset](https://mlsanity.com/paper/2008.10545/) (2020)
- [Weakly Supervised Learning with Side Information for Noisy Labeled Images](https://mlsanity.com/paper/2008.11586/) (2020)
- [Discovering Visual Patterns in Art Collections with Spatially-consistent Feature Learning](https://mlsanity.com/paper/1903.02678/) (2019)
- [Context-Aware Embeddings for Automatic Art Analysis](https://mlsanity.com/paper/1904.04985/) (2019)
- [The iMet Collection 2019 Challenge Dataset](https://mlsanity.com/paper/1906.00901/) (2019)

## Built on by (most cited first, 30 of 40)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Few-Shot Object Detection with Fully Cross-Transformer](https://mlsanity.com/paper/2203.15021/) (2022)
- [The ArtBench Dataset: Benchmarking Generative Models with Artworks](https://mlsanity.com/paper/2206.11404/) (2022)
- [TIPS: Text-Image Pretraining with Spatial awareness](https://mlsanity.com/paper/2410.16512/) (2024)
- [ILIAS: Instance-Level Image retrieval At Scale](https://mlsanity.com/paper/2502.11748/) (2025)
- [Towards Universal Image Embeddings: A Large-Scale Dataset and Challenge for Generic Image Representations](https://mlsanity.com/paper/2309.01858/) (2023)
- [Supervised Masked Knowledge Distillation for Few-Shot Transformers](https://mlsanity.com/paper/2303.15466/) (2023)
- [DiGeo: Discriminative Geometry-Aware Learning for Generalized Few-Shot Object Detection](https://mlsanity.com/paper/2303.09674/) (2023)
- [Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysis](https://mlsanity.com/paper/2304.10278/) (2023)
- [MIEB: Massive Image Embedding Benchmark](https://mlsanity.com/paper/2504.10471/) (2025)
- [TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment](https://mlsanity.com/paper/2604.12012/) (2026)
- [Instance-Level Composed Image Retrieval](https://mlsanity.com/paper/2510.25387/) (2025)
- [Multi-Modal Few-Shot Object Detection with Meta-Learning-Based Cross-Modal Prompting](https://mlsanity.com/paper/2204.07841/) (2022)
- [Solving Masked Jigsaw Puzzles with Diffusion Vision Transformers](https://mlsanity.com/paper/2404.07292/) (2024)
- [1st Place Solution in Google Universal Images Embedding](https://mlsanity.com/paper/2210.08473/) (2022)
- [DEArt: Dataset of European Art](https://mlsanity.com/paper/2211.01226/) (2022)
- [Prototype-based Dataset Comparison](https://mlsanity.com/paper/2309.02401/) (2023)
- [UDON: Universal Dynamic Online distillatioN for generic image representations](https://mlsanity.com/paper/2406.08332/) (2024)
- [Poses of People in Art: A Data Set for Human Pose Estimation in Digital Art History](https://mlsanity.com/paper/2301.05124/) (2023)
- [Same or Not? Enhancing Visual Perception in Vision-Language Models](https://mlsanity.com/paper/2512.23592/) (2025)
- [PuzLM: Solving Jigsaw Puzzles with Sequence-to-Sequence Language Models](https://mlsanity.com/paper/2511.06315/) (2025)
- [3rd Continual Learning Workshop Challenge on Egocentric Category and Instance Level Object Understanding](https://mlsanity.com/paper/2212.06833/) (2022)
- [Benchmarking Vision-Language and Multimodal Large Language Models in Zero-shot and Few-shot Scenarios: A study on Christian Iconography](https://mlsanity.com/paper/2509.18839/) (2025)
- [A Systematic Review of NeurIPS Dataset Management Practices](https://mlsanity.com/paper/2411.00266/) (2024)
- [Global-to-Local or Local-to-Global? Enhancing Image Retrieval with Efficient Local Search and Effective Global Re-ranking](https://mlsanity.com/paper/2509.04351/) (2025)
- [SpatialBoost: Enhancing Visual Representation through Language-Guided Reasoning](https://mlsanity.com/paper/2603.22057/) (2026)
- [6th Place Solution to Google Universal Image Embedding](https://mlsanity.com/paper/2210.09377/) (2022)
- [Efficient and Discriminative Image Feature Extraction for Universal Image Retrieval](https://mlsanity.com/paper/2409.13513/) (2024)
- [Artificial Intelligence for Geometry-Based Feature Extraction, Analysis and Synthesis in Artistic Images: A Survey](https://mlsanity.com/paper/2412.01450/) (2024)
- [Instance-Level Generation for Representation Learning](https://mlsanity.com/paper/2510.09171/) (2025)

## Abstract (arXiv)

This work introduces a dataset for large-scale instance-level recognition in the domain of artworks. The proposed benchmark exhibits a number of different challenges such as large inter-class similarity, long tail distribution, and many classes. We rely on the open access collection of The Met museum to form a large training set of about 224k classes, where each class corresponds to a museum exhibit with photos taken under studio conditions. Testing is primarily performed on photos taken by museum guests depicting exhibits, which introduces a distribution shift between training and testing. Testing is additionally performed on a set of images not related to Met exhibits making the task resemble an out-of-distribution detection problem. The proposed benchmark follows the paradigm of other recent datasets for instance-level recognition on different domains to encourage research on domain independent approaches. A number of suitable approaches are evaluated to offer a testbed for future comparisons. Self-supervised and supervised contrastive learning are effectively combined to train the backbone which is used for non-parametric classification that is shown as a promising direction. Dataset webpage: http://cmp.felk.cvut.cz/met/