# Understanding Art through Multi-Modal Retrieval in Paintings

arXiv: https://arxiv.org/abs/1904.10615 · HTML: https://mlsanity.com/paper/1904.10615/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2019.

## Built on by (most cited first, 1 of 1)

- [Language Does More Than Describe: On The Lack Of Figurative Speech in Text-To-Image Models](https://mlsanity.com/paper/2210.10578/) (2022)

## Abstract (arXiv)

In computer vision, visual arts are often studied from a purely aesthetics perspective, mostly by analysing the visual appearance of an artistic reproduction to infer its style, its author, or its representative features. In this work, however, we explore art from both a visual and a language perspective. Our aim is to bridge the gap between the visual appearance of an artwork and its underlying meaning, by jointly analysing its aesthetics and its semantics. We introduce the use of multi-modal techniques in the field of automatic art analysis by 1) collecting a multi-modal dataset with fine-art paintings and comments, and 2) exploring robust visual and textual representations in artistic images.