# Graph Neural Networks for Knowledge Enhanced Visual Representation of Paintings

arXiv: https://arxiv.org/abs/2105.08190 · HTML: https://mlsanity.com/paper/2105.08190/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 63 references that are in the map.
- First posted on arXiv 17 May 2021.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [GraphSAINT: Graph Sampling Based Inductive Learning Method](https://mlsanity.com/paper/1907.04931/) (2019)
- [Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks](https://mlsanity.com/paper/1905.07953/) (2019)
- [Which Tasks Should Be Learned Together in Multi-task Learning?](https://mlsanity.com/paper/1905.07553/) (2019)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [SIGN: Scalable Inception Graph Neural Networks](https://mlsanity.com/paper/2004.11198/) (2020)
- [UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memory](https://mlsanity.com/paper/1609.02132/) (2016)
- [Neural Style Transfer: A Review](https://mlsanity.com/paper/1705.04058/) (2017)
- [CNN-RNN: A Unified Framework for Multi-label Image Classification](https://mlsanity.com/paper/1604.04573/) (2016)
- [How to Read Paintings: Semantic Art Understanding with Multi-Modal Retrieval](https://mlsanity.com/paper/1810.09617/) (2018)
- [Understanding and Creating Art with AI: Review and Outlook](https://mlsanity.com/paper/2102.09109/) (2021)
- [BAM! The Behance Artistic Media Dataset for Recognition Beyond Photography](https://mlsanity.com/paper/1704.08614/) (2017)
- [BridgeNet: A Continuity-Aware Probabilistic Network for Age Estimation](https://mlsanity.com/paper/1904.03358/) (2019)
- [A Unified View of Multi-Label Performance Measures](https://mlsanity.com/paper/1609.00288/) (2016)
- [The Shape of Art History in the Eyes of the Machine](https://mlsanity.com/paper/1801.07729/) (2018)
- [Weakly Supervised Object Detection in Artworks](https://mlsanity.com/paper/1810.02569/) (2018)
- [Discovering Visual Patterns in Art Collections with Spatially-consistent Feature Learning](https://mlsanity.com/paper/1903.02678/) (2019)
- [OmniArt: Multi-task Deep Learning for Artistic Data Analysis](https://mlsanity.com/paper/1708.00684/) (2017)
- [Context-Aware Embeddings for Automatic Art Analysis](https://mlsanity.com/paper/1904.04985/) (2019)
- [Multitask Painting Categorization by Deep Multibranch Neural Network](https://mlsanity.com/paper/1812.08052/) (2018)
- [Recognizing Characters in Art History Using Deep Learning](https://mlsanity.com/paper/2003.14171/) (2020)
- [HyperLearn: A Distributed Approach for Representation Learning in Datasets With Many Modalities](https://mlsanity.com/paper/1909.09252/) (2019)

## Built on by (most cited first, 9 of 9)

- [Compression ensembles quantify aesthetic complexity and the evolution of visual art](https://mlsanity.com/paper/2205.10271/) (2022)
- [ArtRAG: Retrieval-Augmented Generation with Structured Context for Visual Art Understanding](https://mlsanity.com/paper/2505.06020/) (2025)
- [VL-KGE: Vision-Language Models Meet Knowledge Graph Embeddings](https://mlsanity.com/paper/2603.02435/) (2026)
- [Prototype-Enhanced Hypergraph Learning for Heterogeneous Information Networks](https://mlsanity.com/paper/2309.13092/) (2023)
- [A-MAR: Agent-based Multimodal Art Retrieval for Fine-Grained Artwork Understanding](https://mlsanity.com/paper/2604.19689/) (2026)
- [Art Beyond Semantics: Sheaf-Informed Contrastive Learning for Multi-Relational Representations](https://mlsanity.com/paper/2607.16321/) (2026)
- [CompNet: A Designated Model to Handle Combinations of Images and Designed features](https://mlsanity.com/paper/2209.14454/) (2022)
- [Set2Seq Transformer: Temporal and Position-Aware Set Representations for Sequential Multiple-Instance Learning](https://mlsanity.com/paper/2408.03404/) (2024)
- [MMArt: A Multi-Perspective Multimodal Dataset for Visual Art Understanding](https://mlsanity.com/paper/2608.10706/) (2026)

## Abstract (arXiv)

We propose ArtSAGENet, a novel multimodal architecture that integrates Graph Neural Networks (GNNs) and Convolutional Neural Networks (CNNs), to jointly learn visual and semantic-based artistic representations. First, we illustrate the significant advantages of multi-task learning for fine art analysis and argue that it is conceptually a much more appropriate setting in the fine art domain than the single-task alternatives. We further demonstrate that several GNN architectures can outperform strong CNN baselines in a range of fine art analysis tasks, such as style classification, artist attribution, creation period estimation, and tag prediction, while training them requires an order of magnitude less computational time and only a small amount of labeled data. Finally, through extensive experimentation we show that our proposed ArtSAGENet captures and encodes valuable relational dependencies between the artists and the artworks, surpassing the performance of traditional methods that rely solely on the analysis of visual content. Our findings underline a great potential of integrating visual content and semantics for fine art analysis and curation.