# Formal Analysis of Art: Proxy Learning of Visual Concepts from Style Through Language Models

arXiv: https://arxiv.org/abs/2201.01819 · HTML: https://mlsanity.com/paper/2201.01819/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 35 references that are in the map.
- First posted on arXiv 5 Jan 2022.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [The Shape of Art History in the Eyes of the Machine](https://mlsanity.com/paper/1801.07729/) (2018)
- [DEEP-CARVING: Discovering Visual Attributes by Carving Deep Neural Nets](https://mlsanity.com/paper/1504.04871/) (2015)

## Built on by (most cited first, 7 of 7)

- [StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translation](https://mlsanity.com/paper/2202.12362/) (2022)
- [Multimodal LLMs Can Reason about Aesthetics in Zero-Shot](https://mlsanity.com/paper/2501.09012/) (2025)
- [Have Large Vision-Language Models Mastered Art History?](https://mlsanity.com/paper/2409.03521/) (2024)
- [Docent: A content-based recommendation system to discover contemporary art](https://mlsanity.com/paper/2207.05648/) (2022)
- [Towards mapping the contemporary art world with ArtLM: an art-specific NLP model](https://mlsanity.com/paper/2212.07127/) (2022)
- [The persistence of painting styles](https://mlsanity.com/paper/2511.16695/) (2025)
- [Abstract4D: A Large-Scale Dataset and Framework for Understanding the Visual Language of Abstract Art](https://mlsanity.com/paper/2608.28339/) (2026)

## Abstract (arXiv)

We present a machine learning system that can quantify fine art paintings with a set of visual elements and principles of art. This formal analysis is fundamental for understanding art, but developing such a system is challenging. Paintings have high visual complexities, but it is also difficult to collect enough training data with direct labels. To resolve these practical limitations, we introduce a novel mechanism, called proxy learning, which learns visual concepts in paintings though their general relation to styles. This framework does not require any visual annotation, but only uses style labels and a general relationship between visual concepts and style. In this paper, we propose a novel proxy model and reformulate four pre-existing methods in the context of proxy learning. Through quantitative and qualitative comparison, we evaluate these methods and compare their effectiveness in quantifying the artistic visual concepts, where the general relationship is estimated by language models; GloVe or BERT. The language modeling is a practical and scalable solution requiring no labeling, but it is inevitably imperfect. We demonstrate how the new proxy model is robust to the imperfection, while the other models are sensitively affected by it.