mlsanityData to 14 Oct 2026

Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation

arXiv 2109.05743 ↗First posted 13 Sep 2021cs.CV cs.AI cs.CLcited by 30

Not on the map. Why?

Abstract (arXiv)

Have you ever looked at a painting and wondered what is the story behind it? This work presents a framework to bring art closer to people by generating comprehensive descriptions of fine-art paintings. Generating informative descriptions for artworks, however, is extremely challenging, as it requires to 1) describe multiple aspects of the image such as its style, content, or composition, and 2) provide background and contextual knowledge about the artist, their influences, or the historical period. To address these challenges, we introduce a multi-topic and knowledgeable art description framework, which modules the generated sentences according to three artistic topics and, additionally, enhances each description with external knowledge. The framework is validated through an exhaustive analysis, both quantitative and qualitative, as well as a comparative human evaluation, demonstrating outstanding results in terms of both topic diversity and information veracity.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 38 of its 78 references that are in the map. First posted on arXiv 13 Sep 2021.

Builds on

38 of 78 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  8. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  9. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  10. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  11. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  12. 2018Hierarchical Neural Story Generationcited by 1,193
26 more
  1. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,127
  2. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  3. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  4. 2016Self-critical Sequence Training for Image Captioningcited by 547
  5. 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
  6. 2015Skip-Thought Vectorscited by 322
  7. 2020Evaluation of Text Generation: A Surveycited by 277
  8. 2017Toward Controlled Generation of Textcited by 272
  9. 2018Neural Baby Talkcited by 107
  10. 2018How to Read Paintings: Semantic Art Understanding with Multi-Modal Retrievalcited by 74
  11. 2019Good News, Everyone! Context driven entity-aware captioning for news imagescited by 66
  12. 2013Recognizing Image Stylecited by 64
  13. 2017Towards Diverse and Natural Image Descriptions via a Conditional GANcited by 59
  14. 2018Inverse Cooking: Recipe Generation from Food Imagescited by 47
  15. 2016Image Captioning and Visual Question Answering Based on Attributes and External Knowledgecited by 44
  16. 2017BAM! The Behance Artistic Media Dataset for Recognition Beyond Photographycited by 43
  17. 2020A Dataset and Baselines for Visual Question Answering on Artcited by 38
  18. 2018Weakly Supervised Object Detection in Artworkscited by 27
  19. 2020Show, Recall, and Tell: Image Captioning with Recall Mechanismcited by 27
  20. 2018Entity-aware Image Caption Generationcited by 26
  21. 2017Incorporating Copying Mechanism in Image Captioning for Learning Novel Objectscited by 20
  22. 2019Context-Aware Embeddings for Automatic Art Analysiscited by 16
  23. 2019Pointing Novel Objects in Image Captioningcited by 15
  24. 2019Improving Image Captioning by Leveraging Knowledge Graphscited by 11
  25. 2018Unsupervised Learning of Artistic Styles with Archetypal Style Analysiscited by 10
  26. 2020Demographic Influences on Contemporary Art with Unsupervised Style Embeddingscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 1; 2024: 11; 2025: 6; 2026: 7.

30 papers.

Sort:

By area:

  1. 2024Measuring Style Similarity in Diffusion Modelscited by 109
  2. 2024One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videoscited by 100
  3. 2022SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Textcited by 30
  4. 2023Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysiscited by 18
  5. 2024Diffusion-Based Visual Art Creation: A Survey and New Perspectivescited by 17
  6. 2025Impossible Videoscited by 16
  7. 2022StyleBabel: Artistic Style Tagging and Captioningcited by 12
  8. 2024Towards Artwork Explanation in Large-scale Vision Language Modelscited by 8
  9. 2025ArtRAG: Retrieval-Augmented Generation with Structured Context for Visual Art Understandingcited by 7
  10. 2024Bridging Information Asymmetry in Text-video Retrieval: A Data-centric Approachcited by 5
  11. 2025ArtSeek: Deep artwork understanding via multimodal in-context reasoning and late interaction retrievalcited by 5
  12. 2024Training A Small Emotional Vision Language Model for Visual Art Comprehensioncited by 4
18 more
  1. 2024KALE: An Artwork Image Captioning System Augmented with Heterogeneous Graphcited by 4
  2. 2022How Well Do Vision Transformers (VTs) Transfer To The Non-Natural Image Domain? An Empirical Study Involving Art Classificationcited by 3
  3. 2022Generating image captions with external encyclopedic knowledgecited by 3
  4. 2022Automatic Analysis of Human Body Representations in Western Artcited by 2
  5. 2024Taming CLIP for Fine-grained and Structured Visual Understanding of Museum Exhibitscited by 2
  6. 2024No Annotations for Object Detection in Art through Stable Diffusioncited by 2
  7. 2025CompArt: Operationalizing Aesthetic Alignment in Text-to-Image Generation via Principles of Artcited by 2
  8. 2026VL-KGE: Vision-Language Models Meet Knowledge Graph Embeddingscited by 2
  9. 2024Artificial Intelligence for Geometry-Based Feature Extraction, Analysis and Synthesis in Artistic Images: A Surveycited by 1
  10. 2025TextTIGER: Text-based Intelligent Generation with Entity Prompt Refinement for Text-to-Image Generationcited by 1
  11. 2026HanMoVLM: Large Vision-Language Models for Professional Artistic Painting Evaluationcited by 1
  12. 2026A-MAR: Agent-based Multimodal Art Retrieval for Fine-Grained Artwork Understandingcited by 1
  13. 2026Art Beyond Semantics: Sheaf-Informed Contrastive Learning for Multi-Relational Representationscited by 1
  14. 2024Set2Seq Transformer: Temporal and Position-Aware Set Representations for Sequential Multiple-Instance Learningcited by 0
  15. 2025ImageSet2Text: Describing Sets of Images through Textcited by 0
  16. 2026Art2Mus: Artwork-to-Music Generation via Visual Conditioning and Large-Scale Cross-Modal Alignmentcited by 0
  17. 2026Learning visual representations for compositional analysis of artworks and photographscited by 0
  18. 2026MMArt: A Multi-Perspective Multimodal Dataset for Visual Art Understandingcited by 0