Can Vision Language Models Assess Graphic Design Aesthetics? A Benchmark, Evaluation, and Dataset Perspective
On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment
Abstract (arXiv)
Assessing the aesthetic quality of graphic design is central to visual communication, yet remains underexplored in vision language models (VLMs). We investigate whether VLMs can evaluate design aesthetics in ways comparable to humans. Prior work faces three key limitations: benchmarks restricted to narrow principles and coarse evaluation protocols, a lack of systematic VLM comparisons, and limited training data for model improvement. In this work, we introduce AesEval-Bench, a comprehensive benchmark spanning four dimensions, twelve indicators, and three fully quantifiable tasks: aesthetic judgment, region selection, and precise localization. Then, we systematically evaluate proprietary, open-source, and reasoning-augmented VLMs, revealing clear performance gaps against the nuanced demands of aesthetic assessment. Moreover, we construct a training dataset to fine-tune VLMs for this domain, leveraging human-guided VLM labeling to produce task labels at scale and indicator-grounded reasoning to tie abstract indicators to concrete design regions.Together, our work establishes the first systematic framework for aesthetic quality assessment in graphic design. Our code and dataset will be released at: hrefhttps://github.com/arctanxarc/AesEval-Benchhttps://github.com/arctanxarc/AesEval-Bench
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 44 of its 50 references that are in the map. First posted on arXiv 1 Mar 2026.
Builds on
44 of 50 references.
Closest ancestors · 2
Same direction, newest first
- 2025Multimodal LLMs Can Reason about Aesthetics in Zero-Shot15 Jan
- 2024AesBench: An Expert Benchmark for Multimodal Large Language Models on Image Aesthetics Perception16 Jan
Other references · 42
- 2023GPT-4 Technical Reportcited by 18,132
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Visual Instruction Tuningcited by 7,792
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 284
- 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 267
30 more
- 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 204
- 2024Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Wantcited by 98
- 2021CanvasVAE: Learning to Generate Vector Graphic Documentscited by 93
- 2024A Survey on Benchmarks of Multimodal Large Language Modelscited by 81
- 2025Grounded Reinforcement Learning for Visual Reasoningcited by 80
- 2024ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Areacited by 68
- 2025Grounded Chain-of-Thought for Multimodal Large Language Modelscited by 57
- 2024Visual Agents as Fast and Slow Thinkerscited by 53
- 2024MC-LLaVA: Multi-Concept Personalized Vision-Language Modelcited by 53
- 2016Image Aesthetic Assessment: An Experimental Surveycited by 50
- 2024Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoningcited by 50
- 2025Perceive Anything: Recognize, Explain, Caption, and Segment Anything in Images and Videoscited by 49
- 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learningcited by 46
- 2024Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extractioncited by 44
- 2025Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMscited by 41
- 2025Are Video Models Ready as Zero-Shot Reasoners? An Empirical Study with the MME-CoF Benchmarkcited by 41
- 2024UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmarkcited by 39
- 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipescited by 22
- 2025R-VLM: Region-Aware Vision Language Model for Precise GUI Groundingcited by 20
- 2023DEsignBench: Exploring and Benchmarking DALL-E 3 for Imagining Visual Designcited by 17
- 2024Can GPTs Evaluate Graphic Design Based on Design Principles?cited by 14
- 2024PanoSent: A Panoptic Sextuple Extraction Benchmark for Multimodal Conversational Aspect-based Sentiment Analysiscited by 13
- 2025Dr.V: A Hierarchical Perception-Temporal-Cognition Framework to Diagnose Video Hallucination by Fine-grained Spatial-Temporal Groundingcited by 13
- 2026GENIUS: Generative Fluid Intelligence Evaluation Suitecited by 11
- 2025TRISHUL: Towards Region Identification and Screen Hierarchy Understanding for Large VLM based GUI Agentscited by 9
- 2024DesignProbe: A Graphic Design Benchmark for Multimodal Large Language Modelscited by 7
- 2025Open-ended Hierarchical Streaming Video Understanding with Vision Language Modelscited by 6
- 2025UI-Bench: A Benchmark for Evaluating Design Capabilities of AI Text-to-App Toolscited by 3
- 2020Understanding Aesthetic Evaluation using Deep Learningcited by 2
- 2025VCU-Bridge: Hierarchical Visual Connotation Understanding via Semantic Bridgingcited by 2
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026MME-CoF-Pro: Evaluating Reasoning Coherence in Video Generative Models with Text and Visual Hintscited by 4
- 2026Evaluating Design Video Generation: Metrics for Compositional Fidelitycited by 0
- 2026FigmaTrace: Capturing Creative Nuances in Human Figma Design Workflowscited by 0
- 2026Editable Visual Designcited by 0
- 2026Feed the Panel Dimensions, Not Verdicts: Rubric-Decomposed Fusion of Vision-Language Aesthetic Judgescited by 0
- 2026AUV-Bench: Aesthetic Understanding and Generation Evaluation for User Interfacescited by 0
- 2026AUV-Bench: Aesthetic Understanding and Generation Evaluation for User Interfaces28 Sep 2026
- 2026Feed the Panel Dimensions, Not Verdicts: Rubric-Decomposed Fusion of Vision-Language Aesthetic Judges22 Sep 2026
- 2026Editable Visual Design3 Sep 2026
- 2026FigmaTrace: Capturing Creative Nuances in Human Figma Design Workflows20 Aug 2026
- 2026Evaluating Design Video Generation: Metrics for Compositional Fidelity15 May 2026
- 2026MME-CoF-Pro: Evaluating Reasoning Coherence in Video Generative Models with Text and Visual Hints20 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026Visual Aesthetic Benchmark: Can Frontier Models Judge Beauty?
- 2026TASTE: A Designer-Annotated Multi-Dimensional Preference Dataset for AI-Generated Graphic Design
- 2026Graphic-Design-Bench: A Comprehensive Benchmark for Evaluating AI on Graphic Design Tasks
- 2025VisJudge-Bench: Aesthetics and Quality Assessment of Visualizations
- 2026Evolving to the Aesthetics of a Vision-Language Model
- 2026DesignSense: A Human Preference Dataset and Reward Modeling Framework for Graphic Layout Generation
- 2026Through the PRISM: Principle-Aware, Interpretable, and Multi-Scale Evaluation of Visual Designs
- 2025AesTest: Measuring Aesthetic Intelligence from Perception to Production