mlsanityData to 7 Oct 2026

CLIP-Art: Contrastive Pre-training for Fine-Grained Art Classification

arXiv 2204.14244 ↗First posted 29 Apr 2022cs.CVcited by 65

Not on the map. Why?

Abstract (arXiv)

Existing computer vision research in artwork struggles with artwork's fine-grained attributes recognition and lack of curated annotated datasets due to their costly creation. To the best of our knowledge, we are one of the first methods to use CLIP (Contrastive Language-Image Pre-Training) to train a neural network on a variety of artwork images and text descriptions pairs. CLIP is able to learn directly from free-form art descriptions, or, if available, curated fine-grained labels. Model's zero-shot capability allows predicting accurate natural language description for a given image, without directly optimizing for the task. Our approach aims to solve 2 challenges: instance retrieval and fine-grained artwork attribute recognition. We use the iMet Dataset, which we consider the largest annotated artwork dataset. In this benchmark we achieved competitive results using only self-supervision.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Builds on 21 of its 31 references that are in the map. First posted on arXiv 29 Apr 2022.

Builds on

21 of 31 references.

References · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  8. 2017Squeeze-and-Excitation Networkscited by 4,763
  9. 2014Going Deeper with Convolutionscited by 4,231
  10. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  11. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  12. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
9 more
  1. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  2. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 763
  3. 2019Lookahead Optimizer: k steps forward, 1 step backcited by 333
  4. 2018Learning to Navigate for Fine-grained Classificationcited by 103
  5. 2019Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognitioncited by 95
  6. 2019See Better Before Looking Closer: Weakly Supervised Data Augmentation Network for Fine-Grained Visual Classificationcited by 56
  7. 2017Fine-grained Visual-textual Representation Learningcited by 27
  8. 2019The iMet Collection 2019 Challenge Datasetcited by 10
  9. 2021Exploring Vision Transformers for Fine-grained Classificationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 16; 2024: 25; 2025: 11; 2026: 6.

65 papers.

Sort:

By area:

  1. 2022Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restorationcited by 72
  2. 2023Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Featurescited by 52
  3. 2022Multimodal Fake News Detection via CLIP-Guided Learningcited by 42
  4. 2024CARZero: Cross-Attention Alignment for Radiology Zero-Shot Classificationcited by 40
  5. 2023Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Studycited by 37
  6. 2024Knowledge-Enhanced Dual-stream Zero-shot Composed Image Retrievalcited by 36
  7. 2023Progressive Visual Prompt Learning with Contrastive Feature Re-formationcited by 35
  8. 2023Towards Generalisable Video Moment Retrieval: Visual-Dynamic Injection to Image-Text Pre-Trainingcited by 32
  9. 2023MMSD2.0: Towards a Reliable Multi-modal Sarcasm Detection Systemcited by 31
  10. 2024SeD: Semantic-Aware Discriminator for Image Super-Resolutioncited by 26
  11. 2022Bootstrapping Multi-view Representations for Fake News Detectioncited by 25
  12. 2023Defense-Prefix for Preventing Typographic Attacks on CLIPcited by 24
38 more
  1. 2023Domain-Controlled Prompt Learningcited by 22
  2. 2024Unveiling Typographic Deceptions: Insights of the Typographic Vulnerability in Large Vision-Language Modelcited by 22
  3. 2025Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrievalcited by 18
  4. 2023Multi-modal Fake News Detection on Social Media via Multi-grained Information Fusioncited by 15
  5. 2023LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuningcited by 15
  6. 2025CausalCLIPSeg: Unlocking CLIP's Potential in Referring Medical Image Segmentation with Causal Interventioncited by 15
  7. 2023Mixed Hierarchy Network for Image Restorationcited by 14
  8. 2024A3lign-DFER: Pioneering Comprehensive Dynamic Affective Alignment for Dynamic Facial Expression Recognition with CLIPcited by 14
  9. 2022GAMA: Generative Adversarial Multi-Object Scene Attackscited by 13
  10. 2023FD-Align: Feature Discrimination Alignment for Fine-tuning Pre-Trained Models in Few-Shot Learningcited by 12
  11. 2024Towards Efficient and Effective Text-to-Video Retrieval with Coarse-to-Fine Visual Representation Learningcited by 12
  12. 2023ARTxAI: Explainable Artificial Intelligence Curates Deep Representation Learning for Artistic Images using Fuzzy Techniquescited by 10
  13. 2024MATE: Meet At The Embedding -- Connecting Images with Long Textscited by 10
  14. 2024Have Large Vision-Language Models Mastered Art History?cited by 9
  15. 2023FER-former: Multi-modal Transformer for Facial Expression Recognitioncited by 8
  16. 2025ArtRAG: Retrieval-Augmented Generation with Structured Context for Visual Art Understandingcited by 7
  17. 2024LEMoN: Label Error Detection using Multimodal Neighborscited by 6
  18. 2023Generative AI in Agriculture: Creating Image Datasets Using DALL.E's Advanced Large Language Model Capabilitiescited by 5
  19. 2024Multimodal Metadata Assignment for Cultural Heritage Artifactscited by 5
  20. 2024Aligning Visual Contrastive learning models via Preference Optimizationcited by 4
  21. 2024Impression-CLIP: Contrastive Shape-Impression Embedding for Fontscited by 3
  22. 2024Task2Box: Box Embeddings for Modeling Asymmetric Task Relationshipscited by 3
  23. 2024Probabilistic Vision-Language Representation for Weakly Supervised Temporal Action Localizationcited by 3
  24. 2025WP-CLIP: Leveraging CLIP to Predict W"olfflin's Principles in Visual Artcited by 3
  25. 2025ToxicTextCLIP: Text-Based Poisoning and Backdoor Attacks on CLIP Pre-trainingcited by 3
  26. 2022General Image Descriptors for Open World Image Retrieval using ViT CLIPcited by 2
  27. 2024Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation Modelcited by 2
  28. 2024SOOD-ImageNet: a Large-Scale Dataset for Semantic Out-Of-Distribution Image Classification and Semantic Segmentationcited by 2
  29. 2024Taming CLIP for Fine-grained and Structured Visual Understanding of Museum Exhibitscited by 2
  30. 2025Improving Visual Recommendation on E-commerce Platforms Using Vision-Language Modelscited by 2
  31. 2022ExpNet: A unified network for Expert-Level Classificationcited by 1
  32. 2024State-of-the-Art Fails in the Art of Damage Detectioncited by 1
  33. 2024AFANet: Adaptive Frequency-Aware Network for Weakly-Supervised Few-Shot Semantic Segmentationcited by 1
  34. 2025PPJudge: Towards Human-Aligned Assessment of Artistic Painting Processcited by 1
  35. 2026A-MAR: Agent-based Multimodal Art Retrieval for Fine-Grained Artwork Understandingcited by 1
  36. 2022A Brief Overview of AI Governance for Responsible Machine Learning Systemscited by 0
  37. 2023Leveraging Vision-Language Foundation Models for Fine-Grained Downstream Taskscited by 0
  38. 2023SCRAPS: Speech Contrastive Representations of Acoustic and Phonetic Spacescited by 0

50 most cited of 65 All 65 (JSON)