mlsanityData to 14 Oct 2026

Chart-CoCa: Self-Improving Chart Understanding of Vision LMs via Code-Driven Synthesis and Candidate-Conditioned Answering

arXiv 2508.11975 ↗First posted 16 Aug 2025cs.AIcited by 2

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

also in 1 more
Abstract (arXiv)

Vision Language Models (VLMs) often struggle with chart understanding tasks, particularly in accurate chart description and complex reasoning. Synthetic data generation is a promising solution, while usually facing the challenge of noise labels. To address this challenge, we first introduce a chart synthesis pipeline that generates aligned chart-question-answer triplets through code generation and execution, ensuring the reliability of synthetic data without human intervention. Furthermore, inspired by test-time scaling that increases inference budget and thereby improves performance, we design a candidate-conditioned answering process. The VLM first generates multiple responses per query, and then synthesizes the final answer by contextualizing these candidates. Experiments demonstrate significant improvements, with up to 15.50 points accuracy gain over the initial VLM, in a fully self-improving paradigm without either human-labeled data or external models.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 47 references that are in the map. First posted on arXiv 16 Aug 2025.

Builds on

29 of 47 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation11 Jan
  2. 2024Distill Visual Chart Reasoning Ability from LLMs to MLLMs24 Oct
  3. 2024ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning4 Jan
  4. 2023MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning15 Nov

Other references · 25

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2023Let's Verify Step by Stepcited by 3,789
  3. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  4. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  6. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  7. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  8. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  9. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  10. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  11. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  12. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 454
13 more
  1. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 340
  2. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  3. 2019PlotQA: Reasoning over Scientific Plotscited by 321
  4. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 319
  5. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 307
  6. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  7. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  8. 2022Self-Training: A Surveycited by 86
  9. 2024Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvementcited by 74
  10. 2024ChartInstruct: Instruction Tuning for Chart Comprehension and Reasoningcited by 68
  11. 2024Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Samplingcited by 61
  12. 2024DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflowscited by 43
  13. 2024Self-Correction is More than Refinement: A Learning Framework for Visual and Language Reasoning Taskscited by 32

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratchcited by 10
  2. 2026Consistency Has a Computable Blind Spot: A Commutation Theory of Label-Free Reliability for Vision-Language Figure Readingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models
  2. 2026Chartographer: Counterfactual Chart Generation for Evaluating Vision-Language Models
  3. 2026Chart-RL: Generalized Chart Comprehension via Reinforcement Learning with Verifiable Rewards
  4. 2025In-Depth and In-Breadth: Pre-training Multimodal Language Models Customized for Comprehensive Chart Understanding
  5. 2025Probing the Visualization Literacy of Vision Language Models: the Good, the Bad, and the Ugly
  6. 2026Chart-RL: Policy Optimization Reinforcement Learning for Enhanced Visual Reasoning in Chart Question Answering with Vision Language Models
  7. 2025Judging the Judges: Can Large Vision-Language Models Fairly Evaluate Chart Comprehension and Reasoning?
  8. 2024MultiChartQA: Benchmarking Vision-Language Models on Multi-Chart Problems