# Advancing Comprehensive Aesthetic Insight with Multi-Scale Text-Guided Self-Supervised Learning

arXiv: https://arxiv.org/abs/2412.11952 · HTML: https://mlsanity.com/paper/2412.11952/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 53 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Builds on (most cited first, 12 of 12)

- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://mlsanity.com/paper/2205.01917/) (2022)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [ShareGPT4V: Improving Large Multi-Modal Models with Better Captions](https://mlsanity.com/paper/2311.12793/) (2023)
- [CogVLM: Visual Expert for Pretrained Language Models](https://mlsanity.com/paper/2311.03079/) (2023)
- [Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels](https://mlsanity.com/paper/2312.17090/) (2023)
- [Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Feature](https://mlsanity.com/paper/1505.00855/) (2015)
- [UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmark](https://mlsanity.com/paper/2404.09619/) (2024)
- [CLIP Brings Better Features to Visual Aesthetics Learners](https://mlsanity.com/paper/2307.15640/) (2023)

## Built on by (most cited first, 3 of 3)

- [Venus: Benchmarking and Empowering Multimodal Large Language Models for Aesthetic Guidance and Cropping](https://mlsanity.com/paper/2602.23980/) (2026)
- [AesFormer: Transform Everyday Photos into Beautiful Memories](https://mlsanity.com/paper/2605.22126/) (2026)
- [ShutterMuse: Capture-Time Photography Guidance with MLLMs](https://mlsanity.com/paper/2606.25763/) (2026)

## Abstract (arXiv)

Image Aesthetic Assessment (IAA) is a vital and intricate task that entails analyzing and assessing an image's aesthetic values, and identifying its highlights and areas for improvement. Traditional methods of IAA often concentrate on a single aesthetic task and suffer from inadequate labeled datasets, thus impairing in-depth aesthetic comprehension. Despite efforts to overcome this challenge through the application of Multi-modal Large Language Models (MLLMs), such models remain underdeveloped for IAA purposes. To address this, we propose a comprehensive aesthetic MLLM capable of nuanced aesthetic insight. Central to our approach is an innovative multi-scale text-guided self-supervised learning technique. This technique features a multi-scale feature alignment module and capitalizes on a wealth of unlabeled data in a self-supervised manner to structurally and functionally enhance aesthetic ability. The empirical evidence indicates that accompanied with extensive instruct-tuning, our model sets new state-of-the-art benchmarks across multiple tasks, including aesthetic scoring, aesthetic commenting, and personalized image aesthetic assessment. Remarkably, it also demonstrates zero-shot learning capabilities in the emerging task of aesthetic suggesting. Furthermore, for personalized image aesthetic assessment, we harness the potential of in-context learning and showcase its inherent advantages.