# Measuring and predicting variation in the difficulty of questions about data visualizations

arXiv: https://arxiv.org/abs/2505.08031 · HTML: https://mlsanity.com/paper/2505.08031/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 May 2025.

## Built on by (most cited first, 1 of 1)

- [Using Vision + Language Models to Predict Item Difficulty](https://mlsanity.com/paper/2603.04670/) (2026)

## Abstract (arXiv)

Understanding what is communicated by data visualizations is a critical component of scientific literacy in the modern era. However, it remains unclear why some tasks involving data visualizations are more difficult than others. Here we administered a composite test composed of five widely used tests of data visualization literacy to a large sample of U.S. adults (N=503 participants).We found that items in the composite test spanned the full range of possible difficulty levels, and that our estimates of item-level difficulty were highly reliable. However, the type of data visualization shown and the type of task involved only explained a modest amount of variation in performance across items, relative to the reliability of the estimates we obtained. These results highlight the need for finer-grained ways of characterizing these items that predict the reliable variation in difficulty measured in this study, and that generalize to other tests of data visualization understanding.