# ChartReformer: Natural Language-Driven Chart Image Editing

arXiv: https://arxiv.org/abs/2403.00209 · HTML: https://mlsanity.com/paper/2403.00209/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 1 Mar 2024.

## Builds on (most cited first, 15 of 15)

- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://mlsanity.com/paper/2309.17421/) (2023)
- [HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models](https://mlsanity.com/paper/2310.14566/) (2023)
- [Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding](https://mlsanity.com/paper/2210.03347/) (2022)
- [PlotQA: Reasoning over Scientific Plots](https://mlsanity.com/paper/1909.00997/) (2019)
- [Chart-to-Text: A Large-Scale Benchmark for Chart Summarization](https://mlsanity.com/paper/2203.06486/) (2022)
- [ChartLlama: A Multimodal LLM for Chart Understanding and Generation](https://mlsanity.com/paper/2311.16483/) (2023)
- [DePlot: One-shot visual language reasoning by plot-to-table translation](https://mlsanity.com/paper/2212.10505/) (2022)
- [UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning](https://mlsanity.com/paper/2305.14761/) (2023)
- [MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering](https://mlsanity.com/paper/2212.09662/) (2022)
- [NL4DV: A Toolkit for Generating Analytic Specifications for Data Visualization from Natural Language Queries](https://mlsanity.com/paper/2008.10723/) (2020)
- [ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules](https://mlsanity.com/paper/2304.02173/) (2023)
- [Context-Aware Chart Element Detection](https://mlsanity.com/paper/2305.04151/) (2023)
- [SpaDen : Sparse and Dense Keypoint Estimation for Real-World Chart Understanding](https://mlsanity.com/paper/2308.01971/) (2023)

## Built on by (most cited first, 13 of 13)

- [ChartMoE: Mixture of Diversely Aligned Expert Connector for Chart Understanding](https://mlsanity.com/paper/2409.03277/) (2024)
- [ChartEdit: How Far Are MLLMs From Automating Chart Analysis? Evaluating MLLMs' Capability via Chart Editing](https://mlsanity.com/paper/2505.11935/) (2025)
- [The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective](https://mlsanity.com/paper/2407.08583/) (2024)
- [ChartEditor: A Reinforcement Learning Framework for Robust Chart Editing](https://mlsanity.com/paper/2511.15266/) (2025)
- [Charts Are Not Images: On the Challenges of Scientific Chart Editing](https://mlsanity.com/paper/2512.00752/) (2025)
- [ChartM3: Benchmarking Chart Editing with Multimodal Instructions](https://mlsanity.com/paper/2507.21167/) (2025)
- [ChartPoint: Guiding MLLMs with Grounding Reflection for Chart Reasoning](https://mlsanity.com/paper/2512.00305/) (2025)
- [PlotEdit: Natural Language-Driven Accessible Chart Editing in PDFs via Multimodal LLM Agents](https://mlsanity.com/paper/2501.11233/) (2025)
- [TRACE: Evidence Grounding-Guided Multi-Video Event Understanding and Claim Generation](https://mlsanity.com/paper/2605.16740/) (2026)
- [Aligned Multi-View Scripts for Universal Chart-to-Code Generation](https://mlsanity.com/paper/2604.24559/) (2026)
- [Does It Run and Is That Enough? Revisiting Text-to-Chart Generation with a Multi-Agent Approach](https://mlsanity.com/paper/2506.06175/) (2025)
- [Back2Struct: Making Structured Images Editable Again](https://mlsanity.com/paper/2609.37016/) (2026)
- [ChartRevise: A Dataset and Evaluation Protocol for Exact Chart Editing via Code](https://mlsanity.com/paper/2609.38642/) (2026)

## Abstract (arXiv)

Chart visualizations are essential for data interpretation and communication; however, most charts are only accessible in image format and lack the corresponding data tables and supplementary information, making it difficult to alter their appearance for different application scenarios. To eliminate the need for original underlying data and information to perform chart editing, we propose ChartReformer, a natural language-driven chart image editing solution that directly edits the charts from the input images with the given instruction prompts. The key in this method is that we allow the model to comprehend the chart and reason over the prompt to generate the corresponding underlying data table and visual attributes for new charts, enabling precise edits. Additionally, to generalize ChartReformer, we define and standardize various types of chart editing, covering style, layout, format, and data-centric edits. The experiments show promising results for the natural language-driven chart image editing.