mlsanityData to 7 Oct 2026

ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation

arXiv 2501.06598 ↗First posted 11 Jan 2025cs.AIcited by 88

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in chart understanding tasks. However, interpreting charts with textual descriptions often leads to information loss, as it fails to fully capture the dense information embedded in charts. In contrast, parsing charts into code provides lossless representations that can effectively contain all critical details. Although existing open-source MLLMs have achieved success in chart understanding tasks, they still face two major challenges when applied to chart-to-code tasks: (1) Low executability and poor restoration of chart details in the generated code and (2) Lack of large-scale and diverse training data. To address these challenges, we propose ChartCoder, the first dedicated chart-to-code MLLM, which leverages Code LLMs as the language backbone to enhance the executability of the generated code. Furthermore, we introduce Chart2Code-160k, the first large-scale and diverse dataset for chart-to-code generation, and propose the Snippet-of-Thought (SoT) method, which transforms direct chart-to-code generation data into step-by-step generation. Experiments demonstrate that ChartCoder, with only 7B parameters, surpasses existing open-source MLLMs on chart-to-code benchmarks, achieving superior chart restoration and code excitability. Our code is available at https://github.com/thunlp/ChartCoder.

Key facts. Cited by 88 papers in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 23 of its 49 references that are in the map. First posted on arXiv 11 Jan 2025.

Builds on

23 of 49 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs28 Jun
  2. 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs26 Jun
  3. 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation14 Jun
  4. 2024Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots13 May
  5. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering5 Mar
  6. 2024ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning19 Feb
  7. 2024ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning4 Jan
  8. 2023ChartLlama: A Multimodal LLM for Chart Understanding and Generation27 Nov
  9. 2023MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning15 Nov
  10. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning24 May

Other references · 13

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  2. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  3. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  4. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  5. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  6. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  7. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  8. 2022Chart-to-Text: A Large-Scale Benchmark for Chart Summarizationcited by 166
  9. 2021SciCap: Generating Captions for Scientific Figurescited by 76
  10. 2024TinyChart: Efficient Chart Understanding with Visual Token Merging and Program-of-Thoughts Learningcited by 51
  11. 2024LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steeringcited by 50
  12. 2024Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Modelcited by 21
1 more
  1. 2024HumanEval-V: Benchmarking High-Level Visual Reasoning with Complex Diagrams in Coding Taskscited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 37; 2026: 51.

88 papers.

Sort:

By area:

  1. 2025CoT-Kinetics: A Theoretical Modeling Assessing LRM Reasoning Processcited by 50
  2. 2025ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLMcited by 36
  3. 2025Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedbackcited by 31
  4. 2025HiDe-LLaVA: Hierarchical Decoupling for Continual Instruction Tuning of Multimodal Large Language Modelcited by 29
  5. 2025ChartEdit: How Far Are MLLMs From Automating Chart Analysis? Evaluating MLLMs' Capability via Chart Editingcited by 28
  6. 2025Chart-R1: Chain-of-Thought Supervision and Reinforcement for Advanced Chart Reasonercited by 24
  7. 2025Code2Video: A Code-centric Paradigm for Educational Video Generationcited by 23
  8. 2025Self-Evolving Multi-Agent Systems via Textual Backpropagationcited by 21
  9. 2025Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generationcited by 18
  10. 2025VinciCoder: Unifying Multimodal Code Generation via Coarse-to-fine Visual Reinforcement Learningcited by 18
  11. 2025Factuality Matters: When Image Generation and Editing Meet Structured Visualscited by 16
  12. 2025Large Language Models (LLMs) for Source Code Analysis: applications, models and datasetscited by 15
38 more
  1. 2025SHARE: An SLM-based Hierarchical Action CorREction Assistant for Text-to-SQLcited by 15
  2. 2025ChartMaster: Advancing Chart-to-Code Generation with Real-World Charts and Chart Similarity Reinforcement Learningcited by 15
  3. 2025Draw with Thought: Unleashing Multimodal Reasoning for Scientific Diagram Generationcited by 14
  4. 2025Enhancing Multimodal Continual Instruction Tuning with BranchLoRAcited by 14
  5. 2026HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Bettercited by 14
  6. 2025AstroVisBench: A Code Benchmark for Scientific Computing and Visualization in Astronomycited by 13
  7. 2025VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Modelscited by 13
  8. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  9. 2025ChartReasoner: Code-Driven Modality Bridging for Long-Chain Reasoning in Chart Question Answeringcited by 11
  10. 2025Boosting Chart-to-Code Generation in MLLM via Dual Preference-Guided Refinementcited by 10
  11. 2025ChartM3: Benchmarking Chart Editing with Multimodal Instructionscited by 9
  12. 2025ChartEditor: A Reinforcement Learning Framework for Robust Chart Editingcited by 9
  13. 2025Improved Iterative Refinement for Chart-to-Code Generation via Structured Instructioncited by 8
  14. 2026ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratchcited by 8
  15. 2025ChartGen: Scaling Chart Understanding Via Code-Guided Synthetic Chart Generationcited by 7
  16. 2025JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligencecited by 7
  17. 2026VisRefiner: Learning from Visual Differences for Screenshot-to-Code Generationcited by 7
  18. 2026Scientific Graphics Program Synthesis via Dual Self-Consistency Reinforcement Learningcited by 6
  19. 2025ChartCards: A Chart-Metadata Generation Framework for Multi-Task Chart Understandingcited by 5
  20. 2025TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Taskscited by 5
  21. 2026ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understandingcited by 5
  22. 2025Visual Programmability: A Guide for Code-as-Thought in Chart Understandingcited by 4
  23. 2026MM-ReCoder: Advancing Chart-to-Code Generation with Reinforcement Learning and Self-Correctioncited by 4
  24. 2026Infinity-Parser2 Technical Reportcited by 4
  25. 2025OpusAnimation: Code-Based Dynamic Chart Generationcited by 3
  26. 2025Composition-Grounded Data Synthesis for Visual Reasoningcited by 3
  27. 2025See Less, See Right: Bi-directional Perceptual Shaping For Multimodal Reasoningcited by 3
  28. 2026GeoTikzBridge: Advancing Multimodal Code Generation for Geometric Perception and Reasoningcited by 3
  29. 2026OmniDiagram: Advancing Unified Diagram Code Generation via Visual Interrogation Rewardcited by 3
  30. 2025Multilingual Multimodal Software Developer for Code Generationcited by 2
  31. 2025Chart-CoCa: Self-Improving Chart Understanding of Vision LMs via Code-Driven Synthesis and Candidate-Conditioned Answeringcited by 2
  32. 2026SVRepair: Structured Visual Reasoning for Automated Program Repaircited by 2
  33. 2026Chart Specification: Structural Representations for Incentivizing VLM Reasoning in Chart-to-Code Generationcited by 2
  34. 2026Visual-ERM: Reward Modeling for Visual Equivalencecited by 2
  35. 2026ChartDiff: A Large-Scale Benchmark for Comprehending Pairs of Chartscited by 2
  36. 2026ChemVLR: Prioritizing Reasoning in Perception for Chemical Vision-Language Understandingcited by 2
  37. 2026CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolutioncited by 2
  38. 2026ChartREG++: Towards Benchmarking and Improving Chart Referring Expression Grounding under Diverse referring clues and Multi-Target Referringcited by 2

50 most cited of 88 All 88 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025START: Spatial and Textual Learning for Chart Understanding
  2. 2026RealChart2Code: Advancing Chart-to-Code Generation with Real Data and Multi-Task Evaluation
  3. 2025From Charts to Code: A Hierarchical Benchmark for Multimodal Models
  4. 2026ChartFI: Benchmarking Faithfulness and Insightfulness of Chart Descriptions from Multimodal Large Language Models
  5. 2025Effective Training Data Synthesis for Improving MLLM Chart Understanding
  6. 2025ChartAnchor: Chart Grounding with Structural-Semantic Fidelity
  7. 2023ChartBench: A Benchmark for Complex Visual Reasoning in Charts
  8. 2024Distill Visual Chart Reasoning Ability from LLMs to MLLMs