# Surfacing Visualization Mirages

arXiv: https://arxiv.org/abs/2001.02316 · HTML: https://mlsanity.com/paper/2001.02316/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jan 2020.

## Built on by (most cited first, 17 of 17)

- [Fits and Starts: Enterprise Use of AutoML and the Role of Humans in the Loop](https://mlsanity.com/paper/2101.04296/) (2021)
- [Protecting multimodal large language models against misleading visualizations](https://mlsanity.com/paper/2502.20503/) (2025)
- [Is this chart lying to me? Automating the detection of misleading visualizations](https://mlsanity.com/paper/2508.21675/) (2025)
- [Using Metamorphic Relations to Verify and Enhance Artcode Classification](https://mlsanity.com/paper/2108.02694/) (2021)
- [User Ex Machina : Simulation as a Design Probe in Human-in-the-Loop Text Analytics](https://mlsanity.com/paper/2101.02244/) (2021)
- [Adversarial Attacks on Machine Learning-Aided Visualizations](https://mlsanity.com/paper/2409.02485/) (2024)
- [The Truth, the Whole Truth, and Nothing but the Truth: Automatic Visualization Evaluation from Reconstruction Quality](https://mlsanity.com/paper/2603.16873/) (2026)
- [Navigating the Mirage: A Dual-Path Agentic Framework for Robust Misleading Chart Question Answering](https://mlsanity.com/paper/2603.28583/) (2026)
- [Eliciting Model Steering Interactions from Users via Data and Visual Design Probes](https://mlsanity.com/paper/2310.09314/) (2023)
- [Formative Study for AI-assisted Data Visualization](https://mlsanity.com/paper/2409.06892/) (2024)
- [Visualization Tasks for Unlabeled Graphs](https://mlsanity.com/paper/2504.14115/) (2025)
- [Data Therapist: Eliciting Domain Knowledge from Subject Matter Experts Using Large Language Models](https://mlsanity.com/paper/2505.00455/) (2025)
- [A General Approach to Visualizing Uncertainty in Statistical Graphics](https://mlsanity.com/paper/2508.00937/) (2025)
- [Visualization Biases MLLM's Decision Making in Network Data Tasks](https://mlsanity.com/paper/2511.03617/) (2025)
- [Disentangling Visual and Factual Correctness in LVLMs' Visualization Literacy](https://mlsanity.com/paper/2606.03142/) (2026)
- [Exploring Agentic Approaches for Data Issue Detection and Repair in AI-Assisted Visualization](https://mlsanity.com/paper/2608.21602/) (2026)
- [Vibe Analysis: Exploring LLM Adoption by Data Visualization Practitioners](https://mlsanity.com/paper/2609.31922/) (2026)

## Abstract (arXiv)

Dirty data and deceptive design practices can undermine, invert, or invalidate the purported messages of charts and graphs. These failures can arise silently: a conclusion derived from a particular visualization may look plausible unless the analyst looks closer and discovers an issue with the backing data, visual specification, or their own assumptions. We term such silent but significant failures "visualization mirages". We describe a conceptual model of mirages and show how they can be generated at every stage of the visual analytics process. We adapt a methodology from software testing, "metamorphic testing", as a way of automatically surfacing potential mirages at the visual encoding stage of analysis through modifications to the underlying data and chart specification. We show that metamorphic testing can reliably identify mirages across a variety of chart types with relatively little prior knowledge of the data or the domain.