# Beauty in the Eye of AI: Aligning LLMs and Vision Models with Human Aesthetics in Network Visualization

arXiv: https://arxiv.org/abs/2604.03417 · HTML: https://mlsanity.com/paper/2604.03417/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 20 of its 46 references that are in the map.
- First posted on arXiv 3 Apr 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Diffusion and multimodal model alignment](https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/)

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Data2Vis: Automatic Generation of Data Visualizations Using Sequence to Sequence Recurrent Neural Networks](https://mlsanity.com/paper/1804.03126/) (2018)
- [Generative AI for Visualization: State of the Art and Future Directions](https://mlsanity.com/paper/2404.18144/) (2024)
- [A Deep Generative Model for Graph Layout](https://mlsanity.com/paper/1904.12225/) (2019)
- [DeepDrawing: A Deep Learning Approach to Graph Drawing](https://mlsanity.com/paper/1907.11040/) (2019)
- [Evaluating the Readability of Force Directed Graph Layouts: A Deep Learning Approach](https://mlsanity.com/paper/1808.00703/) (2018)
- [DeepGD: A Deep Learning Framework for Graph Drawing Using GNN](https://mlsanity.com/paper/2106.15347/) (2021)
- [How Aligned are Human Chart Takeaways and LLM Predictions? A Case Study on Bar Charts with Varying Layouts](https://mlsanity.com/paper/2408.06837/) (2024)
- [Graph Neural Networks for Graph Drawing](https://mlsanity.com/paper/2109.10061/) (2021)
- [Deep Neural Network for DrawiNg Networks, (DNN)^2](https://mlsanity.com/paper/2108.03632/) (2021)
- [SmartGD: A GAN-Based Graph Drawing Framework for Diverse Aesthetic Goals](https://mlsanity.com/paper/2206.06434/) (2022)
- [Ask and You Shall Receive (a Graph Drawing): Testing ChatGPT's Potential to Apply Graph Layout Algorithms](https://mlsanity.com/paper/2303.08819/) (2023)
- [Aesthetic Discrimination of Graph Layouts](https://mlsanity.com/paper/1809.01017/) (2018)
- [A Heuristic Approach towards Drawings of Graphs with High Crossing Resolution](https://mlsanity.com/paper/1808.10519/) (2018)
- [Stress-Plus-X (SPX) Graph Layout](https://mlsanity.com/paper/1908.01769/) (2019)
- [Learning to Automate Chart Layout Configurations Using Crowdsourced Paired Comparison](https://mlsanity.com/paper/2101.03680/) (2021)
- [The Perception of Stress in Graph Drawings](https://mlsanity.com/paper/2409.04493/) (2024)
- [A Machine Learning Approach for Predicting Human Preference for Graph Layouts](https://mlsanity.com/paper/2103.03665/) (2021)

## Abstract (arXiv)

Network visualization has traditionally relied on heuristic metrics, such as stress, under the assumption that optimizing them leads to aesthetic and informative layouts. However, no single metric consistently produces the most effective results. A data-driven alternative is to learn from human preferences, where annotators select their favored visualization among multiple layouts of the same graphs. These human-preference labels can then be used to train a generative model that approximates human aesthetic preferences. However, obtaining human labels at scale is costly and time-consuming. As a result, this generative approach has so far been tested only with machine-labeled data. In this paper, we explore the use of large language models (LLMs) and vision models (VMs) as proxies for human judgment. Through a carefully designed user study involving 27 participants, we curated a large set of human preference labels. We used this data both to better understand human preferences and to bootstrap LLM/VM labelers. We show that prompt engineering that combines few-shot examples and diverse input formats, such as image embeddings, significantly improves LLM-human alignment, and additional filtering by the confidence score of the LLM pushes the alignment to human-human levels. Furthermore, we demonstrate that carefully trained VMs can achieve VM-human alignment at a level comparable to that between human annotators. Our results suggest that AI can feasibly serve as a scalable proxy for human labelers.