# The Rational Agent Benchmark for Data Visualization

arXiv: https://arxiv.org/abs/2304.03432 · HTML: https://mlsanity.com/paper/2304.03432/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2023.

## Built on by (most cited first, 10 of 10)

- [A Decision Theoretic Framework for Measuring AI Reliance](https://mlsanity.com/paper/2401.15356/) (2024)
- [SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents](https://mlsanity.com/paper/2603.29139/) (2026)
- [Conformal Prediction and Human Decision Making](https://mlsanity.com/paper/2503.11709/) (2025)
- [Explaining and Improving Information Complementarities in Multi-Agent Decision-making](https://mlsanity.com/paper/2502.06152/) (2025)
- [Designing Algorithmic Recommendations to Achieve Human-AI Complementarity](https://mlsanity.com/paper/2405.01484/) (2024)
- [Visualization for Trust in Machine Learning Revisited: The State of the Field in 2023](https://mlsanity.com/paper/2403.12005/) (2024)
- [Underspecified Human Decision Experiments Considered Harmful](https://mlsanity.com/paper/2401.15106/) (2024)
- [Unexploited Information Value in Human-AI Collaboration](https://mlsanity.com/paper/2411.10463/) (2024)
- [Explanations are a Means to an End: Decision Theoretic Explanation Evaluation](https://mlsanity.com/paper/2506.22740/) (2025)
- [Trust Your Gut: Comparing Human and Machine Inference from Noisy Visualizations](https://mlsanity.com/paper/2407.16871/) (2024)

## Abstract (arXiv)

Understanding how helpful a visualization is from experimental results is difficult because the observed performance is confounded with aspects of the study design, such as how useful the information that is visualized is for the task. We develop a rational agent framework for designing and interpreting visualization experiments. Our framework conceives two experiments with the same setup: one with behavioral agents (human subjects), and the other one with a hypothetical rational agent. A visualization is evaluated by comparing the expected performance of behavioral agents to that of a rational agent under different assumptions. Using recent visualization decision studies from the literature, we demonstrate how the framework can be used to pre-experimentally evaluate the experiment design by bounding the expected improvement in performance from having access to visualizations, and post-experimentally to deconfound errors of information extraction from errors of optimization, among other analyses.