Navigating the Mirage: A Dual-Path Agentic Framework for Robust Misleading Chart Question Answering
On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection
also in 1 more
Abstract (arXiv)
Despite the success of Vision-Language Models (VLMs), misleading charts remain a significant challenge due to their deceptive visual structures and distorted data representations. We present ChartCynics, an agentic dual-path framework designed to unmask visual deception via a "skeptical" reasoning paradigm. Unlike holistic models, ChartCynics decouples perception from verification: a Diagnostic Vision Path captures structural anomalies (e.g., inverted axes) through strategic ROI cropping, while an OCR-Driven Data Path ensures numerical grounding. To resolve cross-modal conflicts, we introduce an Agentic Summarizer optimized via a two-stage protocol: Oracle-Informed SFT for reasoning distillation and Deception-Aware GRPO for adversarial alignment. This pipeline effectively penalizes visual traps and enforces logical consistency. Evaluations on two benchmarks show that ChartCynics achieves 74.43% and 64.55% accuracy, providing an absolute performance boost of ~29% over the Qwen3-VL-8B backbone, outperforming state-of-the-art proprietary models. Our results demonstrate that specialized agentic workflows can grant smaller open-source models superior robustness, establishing a new foundation for trustworthy chart interpretation.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 59 references that are in the map. First posted on arXiv 30 Mar 2026.
Builds on
32 of 59 references.
Closest ancestors · 11
Same direction, newest first
- 2025ChartPoint: Guiding MLLMs with Grounding Reflection for Chart Reasoning29 Nov
- 2025ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answering29 May
- 2025The Plot Thickens: Quantitative Part-by-Part Exploration of MLLM Visualization Literacy3 Apr
- 2025Unmasking Deceptive Visuals: Benchmarking Multimodal Large Language Models on Misleading Chart Question Answering23 Mar
- 2025Benchmarking Visual Language Models on Standardized Visualization Literacy Tests20 Mar
- 2025Chart-HQA: A Benchmark for Hypothetical Question Answering in Charts6 Mar
- 2025Protecting multimodal large language models against misleading visualizations27 Feb
- 2024CHARTOM: A Visual Theory-of-Mind Benchmark for LLMs on Misleading Charts26 Aug
- 2024How Good (Or Bad) Are LLMs at Detecting Misleading Visualizations?24 Jul
- 2024ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answering11 May
- 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering19 Dec
Other references · 21
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
- 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
- 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
- 2019PlotQA: Reasoning over Scientific Plotscited by 317
- 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 314
- 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 314
9 more
- 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
- 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 157
- 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvementcited by 144
- 2019LEAF-QA: Locate, Encode & Attend for Figure Question Answeringcited by 53
- 2024ChartMoE: Mixture of Diversely Aligned Expert Connector for Chart Understandingcited by 44
- 2020Surfacing Visualization Miragescited by 17
- 2024AskChart: Universal Chart Understanding through Textual Enhancementcited by 12
- 2024"I Came Across a Junk": Understanding Design Flaws of Data Visualization from the Public's Perspectivecited by 9
- 2025Remember Past, Anticipate Future: Learning Continual Multimodal Misinformation Detectorscited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025ChartAgent: A Multimodal Agent for Visually Grounded Reasoning in Complex Chart Question Answering
- 2026Chart Deception in Vision-Language Models: From Vulnerability to Mitigation
- 2026Chartographer: Counterfactual Chart Generation for Evaluating Vision-Language Models
- 2025Losing the Plot: How VLM responses degrade on imperfect charts
- 2026Hierarchical Visual Agent: Managing Contexts in Joint Image-Text Space for Advanced Chart Reasoning
- 2025Cognitive Inception: Agentic Reasoning against Visual Deceptions by Injecting Skepticism
- 2024RealCQA-V2: A Diagnostic Benchmark for Structured Visual Entailment over Scientific Charts
- 2025ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models