# Does AI See like Art Historians? Interpreting How Vision Language Models Recognize Artistic Style

arXiv: https://arxiv.org/abs/2603.11024 · HTML: https://mlsanity.com/paper/2603.11024/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 18 of its 41 references that are in the map.
- First posted on arXiv 11 Mar 2026.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 18 of 18)

- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders](https://mlsanity.com/paper/2407.14435/) (2024)
- [CRAFT: Concept Recursive Activation FacTorization for Explainability](https://mlsanity.com/paper/2211.10154/) (2022)
- [Recognizing Image Style](https://mlsanity.com/paper/1311.3715/) (2013)
- [Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Training](https://mlsanity.com/paper/2403.02325/) (2024)
- [A Dataset and Baselines for Visual Question Answering on Art](https://mlsanity.com/paper/2008.12520/) (2020)
- [A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models](https://mlsanity.com/paper/2502.17516/) (2025)
- [Have Large Vision-Language Models Mastered Art History?](https://mlsanity.com/paper/2409.03521/) (2024)
- [Representational Similarity via Interpretable Visual Concepts](https://mlsanity.com/paper/2503.15699/) (2025)
- [Computational Beauty: Aesthetic Judgment at the Intersection of Art and Science](https://mlsanity.com/paper/1410.2488/) (2014)
- [Deep Ensemble Art Style Recognition](https://mlsanity.com/paper/2405.11675/) (2024)
- [Dream Formulations and Deep Neural Networks: Humanistic Themes in the Iconology of the Machine-Learned Image](https://mlsanity.com/paper/1802.01274/) (2018)

## Built on by (most cited first, 3 of 3)

- [Beyond Starry Night: Shortcut-Aware Control-State Planning for Artist-Grounded Text to Image Generation](https://mlsanity.com/paper/2608.06751/) (2026)
- [Understanding How MLLMs Describe Artworks Using Token Activation Maps](https://mlsanity.com/paper/2606.27947/) (2026)
- [ArtMine: Discovering and Formalizing Artistic Processes](https://mlsanity.com/paper/2607.08331/) (2026)

## Abstract (arXiv)

VLMs have become increasingly proficient at a range of computer vision tasks, such as visual question answering and object detection. This includes increasingly strong capabilities in the domain of art, from analyzing artwork to generation of art. In an interdisciplinary collaboration between computer scientists and art historians, we characterize the mechanisms underlying VLMs' ability to predict artistic style and assess the extent to which they align with the criteria art historians use to reason about artistic style. We employ a latent-space decomposition approach to identify concepts that drive art style prediction and conduct quantitative evaluations, causal analysis and assessment by art historians. Our findings indicate that 73% of the extracted concepts are judged by art historians to exhibit a coherent and semantically meaningful visual feature and 90% of concepts used to predict style of a given artwork were judged relevant. In cases where an irrelevant concept was used to successfully predict style, art historians identified possible reasons for its success; for example, the model might "understand" a concept in more formal terms, such as dark/light contrasts.