# Tell Me Without Telling Me: Two-Way Prediction of Visualization Literacy and Visual Attention

arXiv: https://arxiv.org/abs/2508.03713 · HTML: https://mlsanity.com/paper/2508.03713/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 91 references that are in the map.
- First posted on arXiv 22 Jul 2025.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [What do different evaluation metrics tell us about saliency models?](https://mlsanity.com/paper/1604.03605/) (2016)
- [Salient Object Detection: A Benchmark](https://mlsanity.com/paper/1501.02741/) (2015)
- [Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs](https://mlsanity.com/paper/2101.09824/) (2021)
- [Did the Model Understand the Question?](https://mlsanity.com/paper/1805.05492/) (2018)
- [TranSalNet: Towards perceptually relevant visual saliency prediction](https://mlsanity.com/paper/2110.03593/) (2021)
- [Predicting Visual Importance Across Graphic Design Types](https://mlsanity.com/paper/2008.02912/) (2020)
- [Shared Interest: Measuring Human-AI Alignment to Identify Recurring Patterns in Model Behavior](https://mlsanity.com/paper/2107.09234/) (2021)
- [Learning Visual Importance for Graphic Designs and Data Visualizations](https://mlsanity.com/paper/1708.02660/) (2017)
- [Benchmarking Visual Language Models on Standardized Visualization Literacy Tests](https://mlsanity.com/paper/2503.16632/) (2025)
- [Mini-VLAT: A Short and Effective Measure of Visualization Literacy](https://mlsanity.com/paper/2304.07905/) (2023)
- [Adaptive Assessment of Visualization Literacy](https://mlsanity.com/paper/2308.14147/) (2023)
- [Survey on Individual Differences in Visualization](https://mlsanity.com/paper/2002.07950/) (2020)
- [How to evaluate data visualizations across different levels of understanding](https://mlsanity.com/paper/2009.01747/) (2020)
- [Scanpath Prediction on Information Visualisations](https://mlsanity.com/paper/2112.02340/) (2021)
- [Do You See What I See? A Qualitative Study Eliciting High-Level Visualization Comprehension](https://mlsanity.com/paper/2402.15605/) (2024)
- [Data is Personal: Attitudes and Perceptions of Data Visualization in Rural Pennsylvania](https://mlsanity.com/paper/1901.01920/) (2019)
- [Affective Learning Objectives for Communicative Visualizations](https://mlsanity.com/paper/2208.04078/) (2022)
- [TurkEyes: A Web-Based Toolbox for Crowdsourcing Attention Data](https://mlsanity.com/paper/2001.04461/) (2020)
- [Communicative Visualizations as a Learning Problem](https://mlsanity.com/paper/2009.07095/) (2020)
- [Same Data, Diverging Perspectives: The Power of Visualizations to Elicit Competing Interpretations](https://mlsanity.com/paper/2401.09289/) (2024)

## Built on by (most cited first, 1 of 1)

- [Context-Mediated Domain Adaptation in Multi-Agent Sensemaking Systems](https://mlsanity.com/paper/2603.24858/) (2026)

## Abstract (arXiv)

Accounting for individual differences can improve the effectiveness of visualization design. While the role of visual attention in visualization interpretation is well recognized, existing work often overlooks how this behavior varies based on visual literacy levels. Based on data from a 235-participant user study covering three visualization tests (mini-VLAT, CALVI, and SGL), we show that distinct attention patterns in visual data exploration can correlate with participants' literacy levels: While experts (high-scorers) generally show a strong attentional focus, novices (low-scorers) focus less and explore more. We then propose two computational models leveraging these insights: Lit2Sal -- a novel visual saliency model that predicts observer attention given their visualization literacy level, and Sal2Lit -- a model to predict visual literacy from human visual attention data. Our quantitative and qualitative evaluation demonstrates that Lit2Sal outperforms state-of-the-art saliency models with literacy-aware considerations. Sal2Lit predicts literacy with 86% accuracy using a single attention map, providing a time-efficient supplement to literacy assessment that only takes less than a minute. Taken together, our unique approach to consider individual differences in salience models and visual attention in literacy assessments paves the way for new directions in personalized visual data communication to enhance understanding.