mlsanityData to 7 Oct 2026

AesCanvas: A Large-Scale Dataset and Benchmark for Aesthetic Critique and Contextual Suitability

arXiv 2608.26713 ↗First posted 27 Aug 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment

Abstract (arXiv)

Recent advances in Multimodal Large Language Models (MLLMs) have extended Image Aesthetic Assessment (IAA) beyond scalar scores toward interpretable critique and guidance. Yet existing benchmarks mainly assess intrinsic visual quality or fixed domain criteria, leaving open whether an appealing image is appropriate for a specific purpose, audience, cultural setting, or domain convention. We introduce AesCanvas, a unified suite with two complementary components: CritiqueCanvas with 519,136 instruction-response pairs from 54,300 images supports long-form, multi-dimensional critique across photography, painting, and virtual imagery, whereas ContextCanvas with 301 expert-reviewed use scenarios evaluates contextual aesthetic suitability in realistic use scenarios. Under a unified protocol, we evaluate closed-source frontier, open-weight general, and aesthetic-specific MLLMs. Results reveal a clear separation between critique generation and context-sensitive judgment: reference-based lexical and semantic metrics only partially capture critique quality, while aesthetic specialists remain competitive on selected critique metrics yet substantially lag strong general-purpose MLLMs on ContextCanvas. Further analyses show that aesthetic specialization does not reliably transfer to contextual suitability and that model decisions may fail to track or ground themselves in decisive contextual visual cues. These findings establish culturally situated, evidence-grounded suitability as a distinct objective for aesthetic modeling.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 52 references that are in the map. First posted on arXiv 27 Aug 2026.

Builds on

27 of 52 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025The Photographer Eye: Teaching Multimodal Large Language Models to Understand Image Aesthetics like Photographers23 Sep
  2. 2024AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception15 Apr
  3. 2024AesBench: An Expert Benchmark for Multimodal Large Language Models on Image Aesthetics Perception16 Jan

Other references · 24

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  4. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  5. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  6. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
  7. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
  8. 2017NIMA: Neural Image Assessmentcited by 387
  9. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
  10. 2016Photo Aesthetics Ranking Network with Attributes and Content Adaptationcited by 122
  11. 2023AIGCIQA2023: A Large-scale Image Quality Assessment Database for AI Generated Images: from the Perspectives of Quality, Authenticity and Correspondencecited by 61
  12. 2022Personalized Image Aesthetics Assessment with Rich Attributescited by 56
12 more
  1. 2023Towards Artistic Image Aesthetics Assessment: a Large-scale Dataset and a New Methodcited by 48
  2. 2025ArtiMuse: Fine-Grained Image Aesthetics Assessment with Joint Scoring and Expert-Level Understandingcited by 36
  3. 2025UniPercept: Towards Unified Perceptual-Level Image Understanding across Aesthetics, Quality, Structure, and Texturecited by 19
  4. 2025CaptionQA: Is Your Caption as Useful as the Image Itself?cited by 14
  5. 2025Q-SiT: Teaching LMMs for Image Quality Scoring and Interpretingcited by 13
  6. 2025Seeing Culture: A Benchmark for Visual Reasoning and Groundingcited by 8
  7. 2026Seeing Is Believing? A Benchmark for Multimodal Large Language Models on Visual Illusions and Anomaliescited by 8
  8. 2025GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluationcited by 7
  9. 2026Fine-grained Image Aesthetic Assessment: Learning Discriminative Scores from Relative Rankscited by 5
  10. 2026WorldBench: A Challenging and Visually Diverse Multimodal Reasoning Benchmarkcited by 3
  11. 2025Bridging Cognitive Gap: Hierarchical Description Learning for Artistic Image Aesthetics Assessmentcited by 2
  12. 2026ReactBench: A Cause-Driven Benchmark for Multimodal Hallucination via Systematic Evaluationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025AesTest: Measuring Aesthetic Intelligence from Perception to Production
  2. 2026Visual Aesthetic Benchmark: Can Frontier Models Judge Beauty?
  3. 2026Can MLLMs Critique Like Humans? Evaluating Open-Ended Aesthetic Reasoning in Multimodal Large Language Models
  4. 2025AesBiasBench: Evaluating Bias and Alignment in Multimodal Language Models for Personalized Image Aesthetic Assessment
  5. 2026Venus: Benchmarking and Empowering Multimodal Large Language Models for Aesthetic Guidance and Cropping
  6. 2025Unlocking the Essence of Beauty: Advanced Aesthetic Reasoning with Relative-Absolute Policy Optimization
  7. 2026Can Vision Language Models Assess Graphic Design Aesthetics? A Benchmark, Evaluation, and Dataset Perspective
  8. 2025VisJudge-Bench: Aesthetics and Quality Assessment of Visualizations