Weakly Supervised Object Detection in Artworks
Not on the map. Why?
Abstract (arXiv)
We propose a method for the weakly supervised detection of objects in paintings. At training time, only image-level annotations are needed. This, combined with the efficiency of our multiple-instance learning method, enables one to learn new classes on-the-fly from globally annotated databases, avoiding the tedious task of manually marking objects. We show on several databases that dropping the instance-level annotations only yields mild performance losses. We also introduce a new database, IconArt, on which we perform detection experiments on classes that could not be learned on photographs, such as Jesus Child or Saint Sebastian. To the best of our knowledge, these are the first experiments dealing with the automatic (and in our case weakly supervised) detection of iconographic elements in paintings. We believe that such a method is of great benefit for helping art historians to explore large digital databases.
Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Oct 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 7; 2022: 3; 2023: 6; 2024: 5; 2025: 4; 2026: 2.
27 papers.
Sort:
By area:
- 2021Weakly Supervised Object Localization and Detection: A Surveycited by 84
- 2023Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenescited by 68
- 2021Understanding and Creating Art with AI: Review and Outlookcited by 60
- 2021Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generationcited by 30
- 2021Self-supervised object detection from audio-visual correspondencecited by 29
- 2023Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysiscited by 18
- 2023ARTxAI: Explainable Artificial Intelligence Curates Deep Representation Learning for Artistic Images using Fuzzy Techniquescited by 10
- 2021Graph Neural Networks for Knowledge Enhanced Visual Representation of Paintingscited by 9
- 2022DEArt: Dataset of European Artcited by 9
- 2023Poses of People in Art: A Data Set for Human Pose Estimation in Digital Art Historycited by 6
- 2023SniffyArt: The Dataset of Smelling Personscited by 6
- 2021A deep learning approach to clustering visual artscited by 4
15 more
- 2024KALE: An Artwork Image Captioning System Augmented with Heterogeneous Graphcited by 4
- 2022Unsupervised Clustering of Roman Potsherds via Variational Autoencoderscited by 3
- 2022How Well Do Vision Transformers (VTs) Transfer To The Non-Natural Image Domain? An Empirical Study Involving Art Classificationcited by 3
- 2023Transfer Learning for Olfactory Object Detectioncited by 3
- 2021Object Retrieval and Localization in Large Art Collections using Deep Multi-Style Feature Fusion and Iterative Votingcited by 2
- 2024Context-Infused Visual Grounding for Artcited by 2
- 2024No Annotations for Object Detection in Art through Stable Diffusioncited by 2
- 2025Smelly, dense, and spreaded: The Object Detection for Olfactory References (ODOR) datasetcited by 2
- 2025Benchmarking Vision-Language and Multimodal Large Language Models in Zero-shot and Few-shot Scenarios: A study on Christian Iconographycited by 2
- 2026On the Explainability of Vision-Language Models in Art Historycited by 2
- 2024Artificial Intelligence for Geometry-Based Feature Extraction, Analysis and Synthesis in Artistic Images: A Surveycited by 1
- 2024Visual Motif Identification: Elaboration of a Curated Comparative Dataset and Classification Methodscited by 0
- 2025BOOST: Out-of-Distribution-Informed Adaptive Sampling for Bias Mitigation in Stylistic Convolutional Neural Networkscited by 0
- 2025Data Augmentation via Latent Diffusion Models for Detecting Smell-Related Objects in Historical Artworkscited by 0
- 2026High-Resolution Artwork Outpainting with Global Blueprint Guidance and Layout Controlcited by 0
- 2026High-Resolution Artwork Outpainting with Global Blueprint Guidance and Layout Control7 Jul 2026
- 2026On the Explainability of Vision-Language Models in Art History24 Feb 2026
- 2025Benchmarking Vision-Language and Multimodal Large Language Models in Zero-shot and Few-shot Scenarios: A study on Christian Iconography23 Sep 2025
- 2025Data Augmentation via Latent Diffusion Models for Detecting Smell-Related Objects in Historical Artworks18 Sep 2025
- 2025Smelly, dense, and spreaded: The Object Detection for Olfactory References (ODOR) dataset11 Jul 2025
- 2025BOOST: Out-of-Distribution-Informed Adaptive Sampling for Bias Mitigation in Stylistic Convolutional Neural Networks8 Jul 2025
- 2024No Annotations for Object Detection in Art through Stable Diffusion9 Dec 2024
- 2024Artificial Intelligence for Geometry-Based Feature Extraction, Analysis and Synthesis in Artistic Images: A Survey2 Dec 2024
- 2024Visual Motif Identification: Elaboration of a Curated Comparative Dataset and Classification Methods21 Oct 2024
- 2024Context-Infused Visual Grounding for Art16 Oct 2024
- 2024KALE: An Artwork Image Captioning System Augmented with Heterogeneous Graph17 Sep 2024
- 2023SniffyArt: The Dataset of Smelling Persons20 Nov 2023
15 more
- 2023ARTxAI: Explainable Artificial Intelligence Curates Deep Representation Learning for Artistic Images using Fuzzy Techniques29 Aug 2023
- 2023Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysis20 Apr 2023
- 2023Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenes5 Mar 2023
- 2023Transfer Learning for Olfactory Object Detection24 Jan 2023
- 2023Poses of People in Art: A Data Set for Human Pose Estimation in Digital Art History12 Jan 2023
- 2022DEArt: Dataset of European Art2 Nov 2022
- 2022How Well Do Vision Transformers (VTs) Transfer To The Non-Natural Image Domain? An Empirical Study Involving Art Classification9 Aug 2022
- 2022Unsupervised Clustering of Roman Potsherds via Variational Autoencoders14 Mar 2022
- 2021Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation13 Sep 2021
- 2021Object Retrieval and Localization in Large Art Collections using Deep Multi-Style Feature Fusion and Iterative Voting14 Jul 2021
- 2021A deep learning approach to clustering visual arts11 Jun 2021
- 2021Graph Neural Networks for Knowledge Enhanced Visual Representation of Paintings17 May 2021
- 2021Weakly Supervised Object Localization and Detection: A Survey16 Apr 2021
- 2021Self-supervised object detection from audio-visual correspondence13 Apr 2021
- 2021Understanding and Creating Art with AI: Review and Outlook18 Feb 2021
- 2023Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysiscited by 18
- 2024No Annotations for Object Detection in Art through Stable Diffusioncited by 2
- 2025Data Augmentation via Latent Diffusion Models for Detecting Smell-Related Objects in Historical Artworkscited by 0
- 2026High-Resolution Artwork Outpainting with Global Blueprint Guidance and Layout Controlcited by 0
- 2026High-Resolution Artwork Outpainting with Global Blueprint Guidance and Layout Control7 Jul 2026
- 2025Data Augmentation via Latent Diffusion Models for Detecting Smell-Related Objects in Historical Artworks18 Sep 2025
- 2024No Annotations for Object Detection in Art through Stable Diffusion9 Dec 2024
- 2023Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysis20 Apr 2023
- 2024KALE: An Artwork Image Captioning System Augmented with Heterogeneous Graphcited by 4
- 2025Benchmarking Vision-Language and Multimodal Large Language Models in Zero-shot and Few-shot Scenarios: A study on Christian Iconographycited by 2
- 2026On the Explainability of Vision-Language Models in Art Historycited by 2
- 2026On the Explainability of Vision-Language Models in Art History24 Feb 2026
- 2025Benchmarking Vision-Language and Multimodal Large Language Models in Zero-shot and Few-shot Scenarios: A study on Christian Iconography23 Sep 2025
- 2024KALE: An Artwork Image Captioning System Augmented with Heterogeneous Graph17 Sep 2024
- 2021Weakly Supervised Object Localization and Detection: A Surveycited by 84
- 2023Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenescited by 68
- 2021Understanding and Creating Art with AI: Review and Outlookcited by 60
- 2021Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generationcited by 30
- 2021Self-supervised object detection from audio-visual correspondencecited by 29
- 2023ARTxAI: Explainable Artificial Intelligence Curates Deep Representation Learning for Artistic Images using Fuzzy Techniquescited by 10
- 2021Graph Neural Networks for Knowledge Enhanced Visual Representation of Paintingscited by 9
- 2022DEArt: Dataset of European Artcited by 9
- 2023Poses of People in Art: A Data Set for Human Pose Estimation in Digital Art Historycited by 6
- 2023SniffyArt: The Dataset of Smelling Personscited by 6
- 2021A deep learning approach to clustering visual artscited by 4
- 2022Unsupervised Clustering of Roman Potsherds via Variational Autoencoderscited by 3
12 most cited of 20 All 20 (JSON)
- 2025Smelly, dense, and spreaded: The Object Detection for Olfactory References (ODOR) dataset11 Jul 2025
- 2025BOOST: Out-of-Distribution-Informed Adaptive Sampling for Bias Mitigation in Stylistic Convolutional Neural Networks8 Jul 2025
- 2024Artificial Intelligence for Geometry-Based Feature Extraction, Analysis and Synthesis in Artistic Images: A Survey2 Dec 2024
- 2024Visual Motif Identification: Elaboration of a Curated Comparative Dataset and Classification Methods21 Oct 2024
- 2024Context-Infused Visual Grounding for Art16 Oct 2024
- 2023SniffyArt: The Dataset of Smelling Persons20 Nov 2023
- 2023ARTxAI: Explainable Artificial Intelligence Curates Deep Representation Learning for Artistic Images using Fuzzy Techniques29 Aug 2023
- 2023Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenes5 Mar 2023
- 2023Transfer Learning for Olfactory Object Detection24 Jan 2023
- 2023Poses of People in Art: A Data Set for Human Pose Estimation in Digital Art History12 Jan 2023
- 2022DEArt: Dataset of European Art2 Nov 2022
- 2022How Well Do Vision Transformers (VTs) Transfer To The Non-Natural Image Domain? An Empirical Study Involving Art Classification9 Aug 2022
12 newest of 20 All 20 (JSON)