mlsanityData to 7 Oct 2026

Evaluating Prompting Strategies for Chart Question Answering with Large Language Models

arXiv 2603.22288 ↗First posted 3 Mar 2026cs.CL cs.AI cs.LGcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Prompting strategies affect LLM reasoning performance, but their role in chart-based QA remains underexplored. We present a systematic evaluation of four widely used prompting paradigms (Zero-Shot, Few-Shot, Zero-Shot Chain-of-Thought, and Few-Shot Chain-of-Thought) across GPT-3.5, GPT-4, and GPT-4o on the ChartQA dataset. Our framework operates exclusively on structured chart data, isolating prompt structure as the only experimental variable, and evaluates performance using two metrics: Accuracy and Exact Match. Results from 1,200 diverse ChartQA samples show that Few-Shot Chain-of-Thought prompting consistently yields the highest accuracy (up to 78.2%), particularly on reasoning-intensive questions, while Few-Shot prompting improves format adherence. Zero-Shot performs well only with high-capacity models on simpler tasks. These findings provide actionable guidance for selecting prompting strategies in structured data reasoning tasks, with implications for both efficiency and accuracy in real-world applications.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 20 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

16 of 20 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 14

  1. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  2. 2021InfographicVQAcited by 430
  3. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 360
  4. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
  5. 2019PlotQA: Reasoning over Scientific Plotscited by 317
  6. 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 157
  7. 2023ChartGPT: Leveraging LLMs to Generate Charts from Abstract Natural Languagecited by 71
  8. 2024ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answeringcited by 50
  9. 2024VisEval: A Benchmark for Data Visualization in the Era of Large Language Modelscited by 43
  10. 2023PromptBench: A Unified Library for Evaluation of Large Language Modelscited by 42
  11. 2024Tables as Texts or Images: Evaluating the Table Reasoning Ability of LLMs and MLLMscited by 40
  12. 2024Automated Data Visualization from Natural Language via Large Language Models: An Exploratory Studycited by 26
2 more
  1. 2024VProChart: Answering Chart Question through Visual Perception Alignment Agent and Programmatic Solution Reasoningcited by 6
  2. 2025Evaluating LLMs for Visualization Generation and Understandingcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ReasonLab: A Controlled and Auditable Evaluation of Prompting Techniques for Multiple-Choice QA
  2. 2024Question-Analysis Prompting Improves LLM Performance in Reasoning Tasks
  3. 2023An automatically discovered chain-of-thought prompt generalizes to novel models and datasets
  4. 2025TAG-EQA: Text-And-Graph for Event Question Answering via Structured Prompting Strategies
  5. 2024Language Models Benefit from Preparation with Elicited Knowledge
  6. 2025ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering
  7. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
  8. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models