# Trust Your Gut: Comparing Human and Machine Inference from Noisy Visualizations

arXiv: https://arxiv.org/abs/2407.16871 · HTML: https://mlsanity.com/paper/2407.16871/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 73 references that are in the map.
- First posted on arXiv 23 Jul 2024.

## Builds on (most cited first, 20 of 20)

- [Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC](https://mlsanity.com/paper/1507.04544/) (2015)
- [Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI](https://mlsanity.com/paper/1909.02309/) (2019)
- [Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation](https://mlsanity.com/paper/2204.11788/) (2022)
- [AutoML to Date and Beyond: Challenges and Opportunities](https://mlsanity.com/paper/2010.10777/) (2020)
- [Bayesian Workflow](https://mlsanity.com/paper/2011.01808/) (2020)
- [Trust in AutoML: Exploring Information Needs for Establishing Trust in Automated Machine Learning Systems](https://mlsanity.com/paper/2001.06509/) (2020)
- [Role of Human-AI Interaction in Selective Prediction](https://mlsanity.com/paper/2112.06751/) (2021)
- [AutoDS: Towards Human-Centered Automation of Data Science](https://mlsanity.com/paper/2101.05273/) (2021)
- [Human-AI Collaboration: The Effect of AI Delegation on Human Task Performance and Task Satisfaction](https://mlsanity.com/paper/2303.09224/) (2023)
- [Teaching Humans When To Defer to a Classifier via Exemplars](https://mlsanity.com/paper/2111.11297/) (2021)
- [The Rational Agent Benchmark for Data Visualization](https://mlsanity.com/paper/2304.03432/) (2023)
- [How can AI Automate End-to-End Data Science?](https://mlsanity.com/paper/1910.14436/) (2019)
- [Controlling False Discoveries During Interactive Data Exploration](https://mlsanity.com/paper/1612.01040/) (2016)
- [A Bayesian Cognition Approach to Improve Data Visualization](https://mlsanity.com/paper/1901.02949/) (2019)
- [A Bayesian cognition approach for belief updating of correlation judgement through uncertainty visualizations](https://mlsanity.com/paper/2008.00058/) (2020)
- [Bayesian-Assisted Inference from Visualized Data](https://mlsanity.com/paper/2008.00142/) (2020)
- [When do data visualizations persuade? The impact of prior attitudes on learning about correlations from scatterplot visualizations](https://mlsanity.com/paper/2302.03776/) (2023)
- [Visual Belief Elicitation Reduces the Incidence of False Discovery](https://mlsanity.com/paper/2301.12512/) (2023)
- [EVM: Incorporating Model Checking into Exploratory Visual Analysis](https://mlsanity.com/paper/2308.13024/) (2023)
- [The Impact of Elicitation and Contrasting Narratives on Engagement, Recall and Attitude Change with News Articles Containing Data Visualization](https://mlsanity.com/paper/2401.05511/) (2024)

## Abstract (arXiv)

People commonly utilize visualizations not only to examine a given dataset, but also to draw generalizable conclusions about the underlying models or phenomena. Prior research has compared human visual inference to that of an optimal Bayesian agent, with deviations from rational analysis viewed as problematic. However, human reliance on non-normative heuristics may prove advantageous in certain circumstances. We investigate scenarios where human intuition might surpass idealized statistical rationality. In two experiments, we examine individuals' accuracy in characterizing the parameters of known data-generating models from bivariate visualizations. Our findings indicate that, although participants generally exhibited lower accuracy compared to statistical models, they frequently outperformed Bayesian agents, particularly when faced with extreme samples. Participants appeared to rely on their internal models to filter out noisy visualizations, thus improving their resilience against spurious data. However, participants displayed overconfidence and struggled with uncertainty estimation. They also exhibited higher variance than statistical machines. Our findings suggest that analyst gut reactions to visualizations may provide an advantage, even when departing from rationality. These results carry implications for designing visual analytics tools, offering new perspectives on how to integrate statistical models and analyst intuition for improved inference and decision-making. The data and materials for this paper are available at https://osf.io/qmfv6