# Exploring Diagnostic Prompting Approach for Multimodal LLM-based Visual Complexity Assessment: A Case Study of Amazon Search Result Pages

arXiv: https://arxiv.org/abs/2512.00082 · HTML: https://mlsanity.com/paper/2512.00082/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 26 Nov 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 13 of 13)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [A survey of the Vision Transformers and their CNN-Transformer based Variants](https://mlsanity.com/paper/2305.09880/) (2023)
- [HueManity: Probing Fine-Grained Visual Perception in MLLMs](https://mlsanity.com/paper/2506.03194/) (2025)
- [Interpretable and Reliable Detection of AI-Generated Images via Grounded Reasoning in MLLMs](https://mlsanity.com/paper/2506.07045/) (2025)
- [VER-Bench: Evaluating MLLMs on Reasoning with Fine-Grained Visual Evidence](https://mlsanity.com/paper/2508.04852/) (2025)
- [The Future of MLLM Prompting is Adaptive: A Comprehensive Experimental Evaluation of Prompt Engineering Methods for Robust Multimodal Performance](https://mlsanity.com/paper/2504.10179/) (2025)
- [A Comparative Survey of Vision Transformers for Feature Extraction in Texture Analysis](https://mlsanity.com/paper/2406.06136/) (2024)
- [Visually Interpretable Subtask Reasoning for Visual Question Answering](https://mlsanity.com/paper/2505.08084/) (2025)
- [The Gestalt Computational Model by Persistent Homology](https://mlsanity.com/paper/2405.20583/) (2024)
- [Finding Closure: A Closer Look at the Gestalt Law of Closure in Convolutional Neural Networks](https://mlsanity.com/paper/2408.12460/) (2024)
- [POV Learning: Individual Alignment of Multimodal Models using Human Perception](https://mlsanity.com/paper/2405.04443/) (2024)
- [Cognitive Science-Inspired Evaluation of Core Capabilities for Object Understanding in AI](https://mlsanity.com/paper/2503.21668/) (2025)
- [Multimodal LLM Augmented Reasoning for Interpretable Visual Perception Analysis](https://mlsanity.com/paper/2504.12511/) (2025)

## Abstract (arXiv)

This study investigates whether diagnostic prompting can improve Multimodal Large Language Model (MLLM) reliability for visual complexity assessment of Amazon Search Results Pages (SRP). We compare diagnostic prompting with standard gestalt principles-based prompting using 200 Amazon SRP pages and human expert annotations. Diagnostic prompting showed notable improvements in predicting human complexity judgments, with F1-score increasing from 0.031 to 0.297 (+858% relative improvement), though absolute performance remains modest (Cohen's kappa = 0.071). The decision tree revealed that models prioritize visual design elements (badge clutter: 38.6% importance) while humans emphasize content similarity, suggesting partial alignment in reasoning patterns. Failure case analysis reveals persistent challenges in MLLM visual perception, particularly for product similarity and color intensity assessment. Our findings indicate that diagnostic prompting represents a promising initial step toward human-aligned MLLM-based evaluation, though failure cases with consistent human-MLLM disagreement require continued research and refinement in prompting approaches with larger ground truth datasets for reliable practical deployment.