# VisJudge-Bench: Aesthetics and Quality Assessment of Visualizations

arXiv: https://arxiv.org/abs/2510.22373 · HTML: https://mlsanity.com/paper/2510.22373/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Oct 2025.

## Built on by (most cited first, 16 of 16)

- [A Survey of Data Agents: Emerging Paradigm or Overstated Hype?](https://mlsanity.com/paper/2510.23587/) (2025)
- [DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecycle](https://mlsanity.com/paper/2512.04324/) (2025)
- [InfoPO: Information-Driven Policy Optimization for User-Centric Agents](https://mlsanity.com/paper/2603.00656/) (2026)
- [VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing](https://mlsanity.com/paper/2605.15677/) (2026)
- [Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs](https://mlsanity.com/paper/2603.29232/) (2026)
- [ParisKV: Fast and Drift-Robust KV-Cache Retrieval for Long-Context LLMs](https://mlsanity.com/paper/2602.07721/) (2026)
- [ViviDoc: Generating Interactive Documents through Human-Agent Collaboration](https://mlsanity.com/paper/2603.27991/) (2026)
- [DataMagic: Transforming Tabular Data into Data Insight Video](https://mlsanity.com/paper/2606.20388/) (2026)
- [DataMagic: Authoring Data Videos through Declarative Multi-Agent Orchestration](https://mlsanity.com/paper/2609.33403/) (2026)
- [Structured Visualization Design Knowledge for Grounding Generative Reasoning and Situated Feedback](https://mlsanity.com/paper/2512.20306/) (2025)
- [SciFigAlign: Scoring Scientific Figures by Fine-tuned Alignment of Visuals with Manuscript Evidence](https://mlsanity.com/paper/2607.27066/) (2026)
- [SciFigQual-Bench: A Benchmark for Scientific Figure Quality Assessment with Full-Manuscript Context](https://mlsanity.com/paper/2607.27084/) (2026)
- [FinReportBench: Measuring and Improving Institution-Grade Financial Report Generation](https://mlsanity.com/paper/2608.04374/) (2026)
- [MOONWALK: Mediating Operations with Intent-Evidence-Action Alignment Across Junior-Supervisor Review Workflows in Animation/VFX Pre-Production](https://mlsanity.com/paper/2609.10385/) (2026)
- [ViCo: Visual-oriented Coding with Self-Reflection for Chart Replication](https://mlsanity.com/paper/2609.16014/) (2026)
- [Data Agents: Agentic Data Systems](https://mlsanity.com/paper/2609.24137/) (2026)

## Abstract (arXiv)

Visualization, a domain-specific yet widely used form of imagery, is an effective way to turn complex datasets into intuitive insights, and its value depends on whether data are faithfully represented, clearly communicated, and aesthetically designed. However, evaluating visualization quality is challenging: unlike natural images, it requires simultaneous judgment across data encoding accuracy, information expressiveness, and visual aesthetics. Although multimodal large language models (MLLMs) have shown promising performance in aesthetic assessment of natural images, no systematic benchmark exists for measuring their capabilities in evaluating visualizations. To address this, we propose VisJudge-Bench, the first comprehensive benchmark for evaluating MLLMs' performance in assessing visualization aesthetics and quality. It contains 3,090 expert-annotated samples from real-world scenarios, covering single visualizations, multiple visualizations, and dashboards across 32 chart types. Systematic testing on this benchmark reveals that even the most advanced MLLMs (such as GPT-5) still exhibit significant gaps compared to human experts in judgment, with a Mean Absolute Error (MAE) of 0.553 and a correlation with human ratings of only 0.428. To address this issue, we propose VisJudge, a model specifically designed for visualization aesthetics and quality assessment. Experimental results demonstrate that VisJudge significantly narrows the gap with human judgment, reducing the MAE to 0.421 (a 23.9% reduction) and increasing the consistency with human experts to 0.687 (a 60.5% improvement) compared to GPT-5. The benchmark is available at https://github.com/HKUSTDial/VisJudgeBench.