# Is this chart lying to me? Automating the detection of misleading visualizations

arXiv: https://arxiv.org/abs/2508.21675 · HTML: https://mlsanity.com/paper/2508.21675/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 50 references that are in the map.
- First posted on arXiv 29 Aug 2025.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [The VIA Annotation Software for Images, Audio and Video](https://mlsanity.com/paper/1904.10699/) (2019)
- [DePlot: One-shot visual language reasoning by plot-to-table translation](https://mlsanity.com/paper/2212.10505/) (2022)
- [TinyChart: Efficient Chart Understanding with Visual Token Merging and Program-of-Thoughts Learning](https://mlsanity.com/paper/2404.16635/) (2024)
- [How Good (Or Bad) Are LLMs at Detecting Misleading Visualizations?](https://mlsanity.com/paper/2407.17291/) (2024)
- [VizLinter: A Linter and Fixer Framework for Data Visualization](https://mlsanity.com/paper/2108.10299/) (2021)
- [Benchmarking Visual Language Models on Standardized Visualization Literacy Tests](https://mlsanity.com/paper/2503.16632/) (2025)
- [Surfacing Visualization Mirages](https://mlsanity.com/paper/2001.02316/) (2020)
- [Unmasking Deceptive Visuals: Benchmarking Multimodal Large Language Models on Misleading Chart Question Answering](https://mlsanity.com/paper/2503.18172/) (2025)
- ["I Came Across a Junk": Understanding Design Flaws of Data Visualization from the Public's Perspective](https://mlsanity.com/paper/2407.11497/) (2024)
- [Protecting multimodal large language models against misleading visualizations](https://mlsanity.com/paper/2502.20503/) (2025)

## Built on by (most cited first, 6 of 6)

- [When AI reviews science: Can we trust the referee?](https://mlsanity.com/paper/2604.23593/) (2026)
- [ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation](https://mlsanity.com/paper/2601.12983/) (2026)
- [When Visuals Aren't the Problem: Evaluating Vision-Language Models on Misleading Data Visualizations](https://mlsanity.com/paper/2603.22368/) (2026)
- [VizAnchor: Decoding Manipulation Intent from Tampering Visualizations via Dual-Anchor Reasoning](https://mlsanity.com/paper/2608.24535/) (2026)
- [VisAudit: Evaluating Multimodal Agents for Visual Diagnosis and Repair](https://mlsanity.com/paper/2610.02399/) (2026)
- [When Charts Deceive Web Agents: A Paired Benchmark for Misleading Visualization-Grounded Tasks](https://mlsanity.com/paper/2610.12823/) (2026)

## Abstract (arXiv)

Misleading visualizations are a potent driver of misinformation on social media and the web. By violating chart design principles, they distort data and lead readers to draw inaccurate conclusions. Prior work has shown that both humans and multimodal large language models (MLLMs) are frequently deceived by such visualizations. Automatically detecting misleading visualizations and identifying the specific design rules they violate could help protect readers and reduce the spread of misinformation. However, the training and evaluation of AI models has been limited by the absence of large, diverse, and openly available datasets. In this work, we introduce Misviz, a benchmark of 2,604 real-world visualizations annotated with 12 types of misleaders. To support model training, we also create Misviz-synth, a synthetic dataset of 57,665 visualizations generated using Matplotlib and based on real-world data tables. We perform a comprehensive evaluation on both datasets using state-of-the-art MLLMs, rule-based systems, and image-axis classifiers. Our results reveal that the task remains highly challenging. We release Misviz, Misviz-synth, and the accompanying code.