# Context-Aware Embeddings for Automatic Art Analysis

arXiv: https://arxiv.org/abs/1904.04985 · HTML: https://mlsanity.com/paper/1904.04985/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2019.

## Built on by (most cited first, 16 of 16)

- [The Met Dataset: Instance-level Recognition for Artworks](https://mlsanity.com/paper/2202.01747/) (2022)
- [Disembodied Machine Learning: On the Illusion of Objectivity in NLP](https://mlsanity.com/paper/2101.11974/) (2021)
- [Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation](https://mlsanity.com/paper/2109.05743/) (2021)
- [Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysis](https://mlsanity.com/paper/2304.10278/) (2023)
- [GCNBoost: Artwork Classification by Label Propagation through a Knowledge Graph](https://mlsanity.com/paper/2105.11852/) (2021)
- [ARTxAI: Explainable Artificial Intelligence Curates Deep Representation Learning for Artistic Images using Fuzzy Techniques](https://mlsanity.com/paper/2308.15284/) (2023)
- [Graph Neural Networks for Knowledge Enhanced Visual Representation of Paintings](https://mlsanity.com/paper/2105.08190/) (2021)
- [Describing and Organizing Semantic Web and Machine Learning Systems in the SWeMLS-KG](https://mlsanity.com/paper/2303.15113/) (2023)
- [WP-CLIP: Leveraging CLIP to Predict W"olfflin's Principles in Visual Art](https://mlsanity.com/paper/2508.12668/) (2025)
- [Context-Infused Visual Grounding for Art](https://mlsanity.com/paper/2410.12369/) (2024)
- [Studying Illustrations in Manuscripts: An Efficient Deep-Learning Approach](https://mlsanity.com/paper/2601.05269/) (2025)
- [Art Beyond Semantics: Sheaf-Informed Contrastive Learning for Multi-Relational Representations](https://mlsanity.com/paper/2607.16321/) (2026)
- [Built Year Prediction from Buddha Face with Heterogeneous Labels](https://mlsanity.com/paper/2109.00812/) (2021)
- [Saliency-Aware Automatic Buddhas Statue Recognition](https://mlsanity.com/paper/2402.16980/) (2024)
- [ArtContext: Contextualizing Artworks with Open-Access Art History Articles and Wikidata Knowledge through a LoRA-Tuned CLIP Model](https://mlsanity.com/paper/2602.11349/) (2026)
- [Interpreting Contrastive Embeddings in Specific Domains with Fuzzy Rules](https://mlsanity.com/paper/2603.12227/) (2026)

## Abstract (arXiv)

Automatic art analysis aims to classify and retrieve artistic representations from a collection of images by using computer vision and machine learning techniques. In this work, we propose to enhance visual representations from neural networks with contextual artistic information. Whereas visual representations are able to capture information about the content and the style of an artwork, our proposed context-aware embeddings additionally encode relationships between different artistic attributes, such as author, school, or historical period. We design two different approaches for using context in automatic art analysis. In the first one, contextual data is obtained through a multi-task learning model, in which several attributes are trained together to find visual relationships between elements. In the second approach, context is obtained through an art-specific knowledge graph, which encodes relationships between artistic attributes. An exhaustive evaluation of both of our models in several art analysis problems, such as author identification, type classification, or cross-modal retrieval, show that performance is improved by up to 7.3% in art classification and 37.24% in retrieval when context-aware embeddings are used.