mlsanityData to 7 Oct 2026

Navigating the Mirage: A Dual-Path Agentic Framework for Robust Misleading Chart Question Answering

arXiv 2603.28583 ↗First posted 30 Mar 2026cs.CV cs.AI cs.MMcited by 1

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

also in 1 more
Abstract (arXiv)

Despite the success of Vision-Language Models (VLMs), misleading charts remain a significant challenge due to their deceptive visual structures and distorted data representations. We present ChartCynics, an agentic dual-path framework designed to unmask visual deception via a "skeptical" reasoning paradigm. Unlike holistic models, ChartCynics decouples perception from verification: a Diagnostic Vision Path captures structural anomalies (e.g., inverted axes) through strategic ROI cropping, while an OCR-Driven Data Path ensures numerical grounding. To resolve cross-modal conflicts, we introduce an Agentic Summarizer optimized via a two-stage protocol: Oracle-Informed SFT for reasoning distillation and Deception-Aware GRPO for adversarial alignment. This pipeline effectively penalizes visual traps and enforces logical consistency. Evaluations on two benchmarks show that ChartCynics achieves 74.43% and 64.55% accuracy, providing an absolute performance boost of ~29% over the Qwen3-VL-8B backbone, outperforming state-of-the-art proprietary models. Our results demonstrate that specialized agentic workflows can grant smaller open-source models superior robustness, establishing a new foundation for trustworthy chart interpretation.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 59 references that are in the map. First posted on arXiv 30 Mar 2026.

Builds on

32 of 59 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025ChartPoint: Guiding MLLMs with Grounding Reflection for Chart Reasoning29 Nov
  2. 2025ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answering29 May
  3. 2025The Plot Thickens: Quantitative Part-by-Part Exploration of MLLM Visualization Literacy3 Apr
  4. 2025Unmasking Deceptive Visuals: Benchmarking Multimodal Large Language Models on Misleading Chart Question Answering23 Mar
  5. 2025Benchmarking Visual Language Models on Standardized Visualization Literacy Tests20 Mar
  6. 2025Chart-HQA: A Benchmark for Hypothetical Question Answering in Charts6 Mar
  7. 2025Protecting multimodal large language models against misleading visualizations27 Feb
  8. 2024CHARTOM: A Visual Theory-of-Mind Benchmark for LLMs on Misleading Charts26 Aug
  9. 2024How Good (Or Bad) Are LLMs at Detecting Misleading Visualizations?24 Jul
  10. 2024ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answering11 May
  11. 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering19 Dec

Other references · 21

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  6. 2025Qwen3-VL Technical Reportcited by 2,534
  7. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  8. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
  9. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
  10. 2019PlotQA: Reasoning over Scientific Plotscited by 317
  11. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 314
  12. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 314
9 more
  1. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  2. 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 157
  3. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvementcited by 144
  4. 2019LEAF-QA: Locate, Encode & Attend for Figure Question Answeringcited by 53
  5. 2024ChartMoE: Mixture of Diversely Aligned Expert Connector for Chart Understandingcited by 44
  6. 2020Surfacing Visualization Miragescited by 17
  7. 2024AskChart: Universal Chart Understanding through Textual Enhancementcited by 12
  8. 2024"I Came Across a Junk": Understanding Design Flaws of Data Visualization from the Public's Perspectivecited by 9
  9. 2025Remember Past, Anticipate Future: Learning Continual Multimodal Misinformation Detectorscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025ChartAgent: A Multimodal Agent for Visually Grounded Reasoning in Complex Chart Question Answering
  2. 2026Chart Deception in Vision-Language Models: From Vulnerability to Mitigation
  3. 2026Chartographer: Counterfactual Chart Generation for Evaluating Vision-Language Models
  4. 2025Losing the Plot: How VLM responses degrade on imperfect charts
  5. 2026Hierarchical Visual Agent: Managing Contexts in Joint Image-Text Space for Advanced Chart Reasoning
  6. 2025Cognitive Inception: Agentic Reasoning against Visual Deceptions by Injecting Skepticism
  7. 2024RealCQA-V2: A Diagnostic Benchmark for Structured Visual Entailment over Scientific Charts
  8. 2025ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models