# StyleBabel: Artistic Style Tagging and Captioning

arXiv: https://arxiv.org/abs/2203.05321 · HTML: https://mlsanity.com/paper/2203.05321/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 57 references that are in the map.
- First posted on arXiv 10 Mar 2022.

## Builds on (most cited first, 30 of 34)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Unified Vision-Language Pre-Training for Image Captioning and VQA](https://mlsanity.com/paper/1909.11059/) (2019)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [Bottleneck Transformers for Visual Recognition](https://mlsanity.com/paper/2101.11605/) (2021)
- [Universal Style Transfer via Feature Transforms](https://mlsanity.com/paper/1705.08086/) (2017)
- [Meshed-Memory Transformer for Image Captioning](https://mlsanity.com/paper/1912.08226/) (2019)
- [VirTex: Learning Visual Representations from Textual Annotations](https://mlsanity.com/paper/2006.06666/) (2020)
- [Attention on Attention for Image Captioning](https://mlsanity.com/paper/1908.06954/) (2019)
- [Swapping Autoencoder for Deep Image Manipulation](https://mlsanity.com/paper/2007.00653/) (2020)
- [Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Feature](https://mlsanity.com/paper/1505.00855/) (2015)
- [Texture Networks: Feed-forward Synthesis of Textures and Stylized Images](https://mlsanity.com/paper/1603.03417/) (2016)
- [ArtEmis: Affective Language for Visual Art](https://mlsanity.com/paper/2101.07396/) (2021)
- [Exploring the structure of a real-time, arbitrary neural artistic stylization network](https://mlsanity.com/paper/1705.06830/) (2017)
- [How to Read Paintings: Semantic Art Understanding with Multi-Modal Retrieval](https://mlsanity.com/paper/1810.09617/) (2018)
- [Recognizing Image Style](https://mlsanity.com/paper/1311.3715/) (2013)
- [Artistic style transfer for videos](https://mlsanity.com/paper/1604.08610/) (2016)
- [Selective Convolutional Descriptor Aggregation for Fine-Grained Image Retrieval](https://mlsanity.com/paper/1604.04994/) (2016)
- [BAM! The Behance Artistic Media Dataset for Recognition Beyond Photography](https://mlsanity.com/paper/1704.08614/) (2017)

## Built on by (most cited first, 12 of 12)

- [UKnow: A Unified Knowledge Protocol with Multimodal Knowledge Graph Datasets for Reasoning and Vision-Language Pre-Training](https://mlsanity.com/paper/2302.06891/) (2023)
- [HyperNST: Hyper-Networks for Neural Style Transfer](https://mlsanity.com/paper/2208.04807/) (2022)
- [A LoRA is Worth a Thousand Pictures](https://mlsanity.com/paper/2412.12048/) (2024)
- [PARASOL: Parametric Style Control for Diffusion Image Synthesis](https://mlsanity.com/paper/2303.06464/) (2023)
- [DIFF-NST: Diffusion Interleaving For deFormable Neural Style Transfer](https://mlsanity.com/paper/2307.04157/) (2023)
- [NeAT: Neural Artistic Tracing for Beautiful Style Transfer](https://mlsanity.com/paper/2304.05139/) (2023)
- [Training A Small Emotional Vision Language Model for Visual Art Comprehension](https://mlsanity.com/paper/2403.11150/) (2024)
- [ALADIN-NST: Self-supervised disentangled representation learning of artistic style through Neural Style Transfer](https://mlsanity.com/paper/2304.05755/) (2023)
- [Online In-Context Distillation for Low-Resource Vision Language Models](https://mlsanity.com/paper/2510.18117/) (2025)
- [Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysis](https://mlsanity.com/paper/2408.04909/) (2024)
- [M-ArtAgent: Evidence-Based Multimodal Agent for Implicit Art Influence Discovery](https://mlsanity.com/paper/2604.07468/) (2026)
- [MegaStyle++: Scaling Image Style Space through Hierarchical Style Definition](https://mlsanity.com/paper/2609.01423/) (2026)

## Abstract (arXiv)

We present StyleBabel, a unique open access dataset of natural language captions and free-form tags describing the artistic style of over 135K digital artworks, collected via a novel participatory method from experts studying at specialist art and design schools. StyleBabel was collected via an iterative method, inspired by `Grounded Theory': a qualitative approach that enables annotation while co-evolving a shared language for fine-grained artistic style attribute description. We demonstrate several downstream tasks for StyleBabel, adapting the recent ALADIN architecture for fine-grained style similarity, to train cross-modal embeddings for: 1) free-form tag generation; 2) natural language description of artistic style; 3) fine-grained text search of style. To do so, we extend ALADIN with recent advances in Visual Transformer (ViT) and cross-modal representation learning, achieving a state of the art accuracy in fine-grained style retrieval.