# Vision Transformer attention alignment with human visual perception in aesthetic object evaluation

arXiv: https://arxiv.org/abs/2507.17616 · HTML: https://mlsanity.com/paper/2507.17616/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 83 references that are in the map.
- First posted on arXiv 23 Jul 2025.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [Attention Mechanisms in Computer Vision: A Survey](https://mlsanity.com/paper/2111.07624/) (2021)
- [What do different evaluation metrics tell us about saliency models?](https://mlsanity.com/paper/1604.03605/) (2016)
- [Are Convolutional Neural Networks or Transformers more like human vision?](https://mlsanity.com/paper/2105.07197/) (2021)
- [How Does Attention Work in Vision Transformers? A Visual Analytics Attempt](https://mlsanity.com/paper/2303.13731/) (2023)
- [Self-attention in Vision Transformers Performs Perceptual Grouping, Not Attention](https://mlsanity.com/paper/2303.01542/) (2023)
- [Do Saliency Models Detect Odd-One-Out Targets? New Datasets and Evaluations](https://mlsanity.com/paper/2005.06583/) (2020)
- [Supervised mid-level features for word image representation](https://mlsanity.com/paper/1410.5224/) (2014)
- [Emergence of Human-Like Attention in Self-Supervised Vision Transformers: an eye-tracking study](https://mlsanity.com/paper/2410.22768/) (2024)

## Built on by (most cited first, 2 of 2)

- [A Saccade-inspired Approach to Image Classification using Vision Transformer Attention Maps](https://mlsanity.com/paper/2603.09613/) (2026)
- [Cognitive Alignment At No Cost: Inducing Human Attention Biases For Interpretable Vision Transformers](https://mlsanity.com/paper/2604.20027/) (2026)

## Abstract (arXiv)

Visual attention mechanisms play a crucial role in human perception and aesthetic evaluation. Recent advances in Vision Transformers (ViTs) have demonstrated remarkable capabilities in computer vision tasks, yet their alignment with human visual attention patterns remains underexplored, particularly in aesthetic contexts. This study investigates the correlation between human visual attention and ViT attention mechanisms when evaluating handcrafted objects. We conducted an eye-tracking experiment with 30 participants (9 female, 21 male, mean age 24.6 years) who viewed 20 artisanal objects comprising basketry bags and ginger jars. Using a Pupil Labs eye-tracker, we recorded gaze patterns and generated heat maps representing human visual attention. Simultaneously, we analyzed the same objects using a pre-trained ViT model with DINO (Self-DIstillation with NO Labels), extracting attention maps from each of the 12 attention heads. We compared human and ViT attention distributions using Kullback-Leibler divergence across varying Gaussian parameters (sigma=0.1 to 3.0). Statistical analysis revealed optimal correlation at sigma=2.4 +-0.03, with attention head #12 showing the strongest alignment with human visual patterns. Significant differences were found between attention heads, with heads #7 and #9 demonstrating the greatest divergence from human attention (p< 0.05, Tukey HSD test). Results indicate that while ViTs exhibit more global attention patterns compared to human focal attention, certain attention heads can approximate human visual behavior, particularly for specific object features like buckles in basketry items. These findings suggest potential applications of ViT attention mechanisms in product design and aesthetic evaluation, while highlighting fundamental differences in attention strategies between human perception and current AI models.