# Learning visual representations for compositional analysis of artworks and photographs

arXiv: https://arxiv.org/abs/2608.06142 · HTML: https://mlsanity.com/paper/2608.06142/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 51 references that are in the map.
- First posted on arXiv 6 Aug 2026.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [ArtEmis: Affective Language for Visual Art](https://mlsanity.com/paper/2101.07396/) (2021)
- [Generalization and Robustness Implications in Object-Centric Learning](https://mlsanity.com/paper/2107.00637/) (2021)
- [AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception](https://mlsanity.com/paper/2404.09624/) (2024)
- [Towards Artistic Image Aesthetics Assessment: a Large-scale Dataset and a New Method](https://mlsanity.com/paper/2303.15166/) (2023)
- [Temporally Consistent Object-Centric Learning by Contrasting Slots](https://mlsanity.com/paper/2412.14295/) (2024)
- [Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation](https://mlsanity.com/paper/2109.05743/) (2021)
- [Grid Anchor based Image Cropping: A New Benchmark and An Efficient Model](https://mlsanity.com/paper/1909.08989/) (2019)
- [SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels](https://mlsanity.com/paper/2410.08822/) (2024)
- [Semantic Segmentation in Art Paintings](https://mlsanity.com/paper/2203.03238/) (2022)
- [Image Composition Assessment with Saliency-augmented Multi-pattern Pooling](https://mlsanity.com/paper/2104.03133/) (2021)
- [APDDv2: Aesthetics of Paintings and Drawings Dataset with Artist Labeled Scores and Comments](https://mlsanity.com/paper/2411.08545/) (2024)
- [LAPIS: A novel dataset for personalized image aesthetic assessment](https://mlsanity.com/paper/2504.07670/) (2025)
- [Understanding Aesthetics with Language: A Photo Critique Dataset for Aesthetic Assessment](https://mlsanity.com/paper/2206.08614/) (2022)
- [Spatial-Semantic Collaborative Cropping for User Generated Content](https://mlsanity.com/paper/2401.08086/) (2024)
- [The Photographer Eye: Teaching Multimodal Large Language Models to Understand Image Aesthetics like Photographers](https://mlsanity.com/paper/2509.18582/) (2025)
- [PhotoFramer: Multi-modal Image Composition Instruction](https://mlsanity.com/paper/2512.00993/) (2025)
- [Image Aesthetics Assessment Using Graph Attention Network](https://mlsanity.com/paper/2206.12869/) (2022)
- [Relational Object-Centric Actor-Critic](https://mlsanity.com/paper/2310.17178/) (2023)
- [Self-supervised Photographic Image Layout Representation Learning](https://mlsanity.com/paper/2403.03740/) (2024)
- [Enhancing Historical Image Retrieval with Compositional Cues](https://mlsanity.com/paper/2403.14287/) (2024)

## Abstract (arXiv)

Composition, the deliberate arrangement of visual elements, is central to how meaning, emotion, and aesthetic quality are conveyed in artwork, yet it remains among the least formalized dimensions of visual understanding. Prior work highlights a persistent gap in learning meaningful compositional representations, attributing it to semantic bias and suggesting that human-inspired approaches may be key. We compare two parallel paradigms for composition analysis: a human-inspired method grounded in perceptual grouping, and fine-tuned foundation models enabled by recent large-scale compositional datasets. The human-inspired approach uses object-centric models for region-level decomposition and a graph attention network to capture spatial relationships between elements. Both paradigms are evaluated on composition score/category prediction, compositional image retrieval, and visual saliency detection. With frozen encoders, the human-inspired method achieves competitive performance while remaining interpretable. When sufficient data enables fine-tuning, large self-supervised models outperform significantly, but at the cost of interpretability and cross-domain generalization. Code and pre-trained models are available on GitHub.