# VisAudit: Evaluating Multimodal Agents for Visual Diagnosis and Repair

arXiv: https://arxiv.org/abs/2610.02399 · HTML: https://mlsanity.com/paper/2610.02399/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 16 references that are in the map.
- First posted on arXiv 1 Oct 2026.

## Builds on (most cited first, 12 of 12)

- [ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation](https://mlsanity.com/paper/2406.09961/) (2024)
- [ChartEdit: How Far Are MLLMs From Automating Chart Analysis? Evaluating MLLMs' Capability via Chart Editing](https://mlsanity.com/paper/2505.11935/) (2025)
- [Unmasking Deceptive Visuals: Benchmarking Multimodal Large Language Models on Misleading Chart Question Answering](https://mlsanity.com/paper/2503.18172/) (2025)
- [From Charts to Code: A Hierarchical Benchmark for Multimodal Models](https://mlsanity.com/paper/2510.17932/) (2025)
- [ChartEditor: A Reinforcement Learning Framework for Robust Chart Editing](https://mlsanity.com/paper/2511.15266/) (2025)
- [ChartEditBench: Evaluating Grounded Multi-Turn Chart Editing in Multimodal Language Models](https://mlsanity.com/paper/2602.15758/) (2026)
- [ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding](https://mlsanity.com/paper/2603.27064/) (2026)
- [Is this chart lying to me? Automating the detection of misleading visualizations](https://mlsanity.com/paper/2508.21675/) (2025)
- [ChartE3: A Comprehensive Benchmark for End-to-End Chart Editing](https://mlsanity.com/paper/2601.21694/) (2026)
- [VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback?](https://mlsanity.com/paper/2608.10408/) (2026)
- [MisVisFix: An Interactive Dashboard for Detecting, Explaining, and Correcting Misleading Visualizations using Large Language Models](https://mlsanity.com/paper/2508.04679/) (2025)
- [Evaluating Self-Correcting Vision Agents Through Quantitative and Qualitative Metrics](https://mlsanity.com/paper/2601.11637/) (2026)

## Abstract (arXiv)

Multimodal agents are increasingly used for data visualization tasks but remain limited in autonomous review. Unlike humans, they may fail to recognize when a visualization is incorrect, determine what to change, repair it without disrupting correct content, and verify whether the intervention succeeded. Existing benchmarks largely evaluate predefined individual capabilities such as chart generation, instruction-guided editing, or defect detection, and therefore do not capture this gap in autonomous review. We introduce VisAudit, a benchmark for evaluating visualization diagnosis, repair, and verification. Given a rendered chart and configurable auxiliary evidence, including its source data table, intended text summary, and visualization code, an agent iteratively diagnoses potential defects, modifies and executes visualization code, inspects execution and visual feedback, and determines when no further intervention is needed. VisAudit defines three tracks spanning diagnosed repair, autonomous repair, and open-world verification, and contains 1,900 flawed instances across 21 chart types and 10 flaw categories, together with 300 initially correct charts. We construct the benchmark through controlled perturbations of validated source visualizations, with systematic verification and human-aligned quality control to ensure that injected defects are well-defined and recoverable from the available evidence. Experiments with leading multimodal models reveal a substantial gap from reliable autonomous review: the strongest evaluated model fully recovers only 47.4% of flawed charts in the autonomous-repair setting.