# Can MLLMs Critique Like Humans? Evaluating Open-Ended Aesthetic Reasoning in Multimodal Large Language Models

arXiv: https://arxiv.org/abs/2606.29689 · HTML: https://mlsanity.com/paper/2606.29689/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 34 of its 50 references that are in the map.
- First posted on arXiv 29 Jun 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [Image editing and quality assessment](https://mlsanity.com/mllm/image-editing-and-quality-assessment/)

## Builds on (most cited first, 30 of 34)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators](https://mlsanity.com/paper/2404.04475/) (2024)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels](https://mlsanity.com/paper/2312.17090/) (2023)
- [MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://mlsanity.com/paper/1909.02622/) (2019)
- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision](https://mlsanity.com/paper/2309.14181/) (2023)
- [Why We Need New Evaluation Metrics for NLG](https://mlsanity.com/paper/1707.06875/) (2017)
- [Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics](https://mlsanity.com/paper/2006.06264/) (2020)
- [Photo Aesthetics Ranking Network with Attributes and Content Adaptation](https://mlsanity.com/paper/1606.01621/) (2016)
- [AesBench: An Expert Benchmark for Multimodal Large Language Models on Image Aesthetics Perception](https://mlsanity.com/paper/2401.08276/) (2024)
- [UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmark](https://mlsanity.com/paper/2404.09619/) (2024)
- [Understanding Aesthetics with Language: A Photo Critique Dataset for Aesthetic Assessment](https://mlsanity.com/paper/2206.08614/) (2022)
- [The Photographer Eye: Teaching Multimodal Large Language Models to Understand Image Aesthetics like Photographers](https://mlsanity.com/paper/2509.18582/) (2025)
- [Aesthetic Image Captioning From Weakly-Labelled Photographs](https://mlsanity.com/paper/1908.11310/) (2019)
- [AesBiasBench: Evaluating Bias and Alignment in Multimodal Language Models for Personalized Image Aesthetic Assessment](https://mlsanity.com/paper/2509.11620/) (2025)
- [Few-Shot Adversarial Low-Rank Fine-Tuning of Vision-Language Models](https://mlsanity.com/paper/2505.15130/) (2025)
- [VISTA: Vision-Language Inference for Training-Free Stock Time-Series Analysis](https://mlsanity.com/paper/2505.18570/) (2025)

## Built on by (most cited first, 2 of 2)

- [Same Game, Different Story: A Minimal Conservative Strategic Robustness Benchmark for Large Language Model Agents](https://mlsanity.com/paper/2607.19670/) (2026)
- [CircuitSteer: Geometrically Aligned Multi-Layer Steering via Sparse Autoencoder Circuits](https://mlsanity.com/paper/2608.05732/) (2026)

## Abstract (arXiv)

Open-ended aesthetic critique is a challenge for multimodal large language models (MLLMs): it has no single correct answer, and most aesthetic evaluation measures models against numeric scores rather than the written critiques people actually give. We ask whether MLLM critiques are close to human ones, scoring eight open-weight MLLMs from 7B to 397B, plus GPT-5.5, against multiple ranked human critiques for each of 1,227 textttr/photocritique posts under eight prompt conditions. Reference-based similarity gives a misleading picture. In absolute terms the stricter lexical and learned metrics align only weakly with human critiques while a coarse embedding cosine reports broad topical overlap, yet requesting shorter critiques raises those scores and withholding the image barely changes them: the similarity reflects length, the post text, and a stable critiquing style more than image-specific observation. An LLM judge sharpens the question rather than settling it: in the primary condition all four judges prefer the frontier models' critiques to the human ones, but on the 7--8B models they diverge wildly, from 9% to 81% preference on identical pairs. Asked instead how similar each pair is in substance, those judges and two human annotators agree, rating every model between 1.81 and 2.59 on a 1--5 scale, close to ``mostly different''. Behaviorally, the models diverge in ways the scores do not surface: they cover nearly every aesthetic aspect where humans are selective and repeat themselves across critiques of one photo, even when prompted to write at human length. We argue that reference-based similarity rewards a fluent, comprehensive critique style rather than the selectivity and specificity of human critique.