mlsanityData to 14 Oct 2026

OneChart: Purify the Chart Structural Extraction via One Auxiliary Token

arXiv 2404.09987 ↗First posted 15 Apr 2024cs.CVcited by 39

Not on the map. Why?

Abstract (arXiv)

Chart parsing poses a significant challenge due to the diversity of styles, values, texts, and so forth. Even advanced large vision-language models (LVLMs) with billions of parameters struggle to handle such tasks satisfactorily. To address this, we propose OneChart: a reliable agent specifically devised for the structural extraction of chart information. Similar to popular LVLMs, OneChart incorporates an autoregressive main body. Uniquely, to enhance the reliability of the numerical parts of the output, we introduce an auxiliary token placed at the beginning of the total tokens along with an additional decoder. The numerically optimized (auxiliary) token allows subsequent tokens for chart parsing to capture enhanced numerical features through causal attention. Furthermore, with the aid of the auxiliary token, we have devised a self-evaluation mechanism that enables the model to gauge the reliability of its chart parsing results by providing confidence scores for the generated content. Compared to current state-of-the-art (SOTA) chart parsing models, e.g., DePlot, ChartVLM, ChartAst, OneChart significantly outperforms in Average Precision (AP) for chart structural extraction across multiple public benchmarks, despite enjoying only 0.2 billion parameters. Moreover, as a chart parsing agent, it also brings 10%+ accuracy gains for the popular LVLM (LLaVA-1.6) in the downstream ChartQA benchmark.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Builds on 33 of its 46 references that are in the map. First posted on arXiv 15 Apr 2024.

Builds on

33 of 46 references.

References · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  7. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  8. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  9. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  10. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 434
  11. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  12. 2019PlotQA: Reasoning over Scientific Plotscited by 321
21 more
  1. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
  2. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 245
  3. 2019What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysiscited by 191
  4. 2021FeTaQA: Free-form Table Question Answeringcited by 175
  5. 2022Chart-to-Text: A Large-Scale Benchmark for Chart Summarizationcited by 169
  6. 2023mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understandingcited by 168
  7. 2023ChartLlama: A Multimodal LLM for Chart Understanding and Generationcited by 163
  8. 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 160
  9. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 160
  10. 2023MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuningcited by 150
  11. 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derenderingcited by 133
  12. 2024ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoningcited by 128
  13. 2023Vary: Scaling up the Vision Vocabulary for Large Vision-Language Modelscited by 84
  14. 2024ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuningcited by 79
  15. 2020Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Modelcited by 78
  16. 2023ChatSpot: Bootstrapping Multimodal LLMs via Precise Referring Instruction Tuningcited by 65
  17. 2022OpenCQA: Open-ended Question Answering with Chartscited by 61
  18. 2023Merlin:Empowering Multimodal LLMs with Foresight Mindscited by 46
  19. 2024Small Language Model Meets with Reinforced Vision Vocabularycited by 43
  20. 2023Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checkingcited by 35
  21. 2019Data Extraction from Charts via Single Deep Neural Networkcited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 12; 2026: 16.

39 papers.

Sort:

By area:

  1. 2025DeepSeek-OCR: Contexts Optical Compressioncited by 183
  2. 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 180
  3. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 173
  4. 2024Ovis: Structural Embedding Alignment for Multimodal Large Language Modelcited by 169
  5. 2024DreamBench++: A Human-Aligned Benchmark for Personalized Image Generationcited by 131
  6. 2026DeepSeek-OCR 2: Visual Causal Flowcited by 74
  7. 2025SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversioncited by 53
  8. 2024TinyChart: Efficient Chart Understanding with Visual Token Merging and Program-of-Thoughts Learningcited by 52
  9. 2024ChartMoE: Mixture of Diversely Aligned Expert Connector for Chart Understandingcited by 46
  10. 2026STEP3-VL-10B Technical Reportcited by 32
  11. 2024Slow Perception: Let's Perceive Geometric Figures Step-by-stepcited by 24
  12. 2026PaddleOCR-VL-1.6: Expanding the Frontier of Document Parsing with Under-Optimized Region Refinement and Progressive Post-Trainingcited by 21
27 more
  1. 2025Effective Training Data Synthesis for Improving MLLM Chart Understandingcited by 20
  2. 2024The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspectivecited by 15
  3. 2025Perception in Reflectioncited by 13
  4. 2025ChartSketcher: Reasoning with Multimodal Feedback and Reflection for Chart Understandingcited by 13
  5. 2024StrucTexTv3: An Efficient Vision-Language Model for Text-rich Image Perception, Comprehension, and Beyondcited by 11
  6. 2026Youtu-Parsing: Perception, Structuring and Recognition via High-Parallelism Decodingcited by 10
  7. 2026SketchVL: Policy Optimization via Fine-Grained Credit Assignment for Chart Understanding and Morecited by 7
  8. 2024VProChart: Answering Chart Question through Visual Perception Alignment Agent and Programmatic Solution Reasoningcited by 6
  9. 2025Chart-HQA: A Benchmark for Hypothetical Question Answering in Chartscited by 6
  10. 2025Reasoning-OCR: Can Large Multimodal Models Solve Complex Logical Reasoning Problems from OCR Cues?cited by 6
  11. 2025ChartPoint: Guiding MLLMs with Grounding Reflection for Chart Reasoningcited by 6
  12. 2026Visual Self-Refine: A Pixel-Guided Paradigm for Accurate Chart Parsingcited by 5
  13. 2025ChartAgent: A Chart Understanding Framework with Tool Integrated Reasoningcited by 4
  14. 2026Beyond Human Annotation: Recent Advances in Data Generation Methods for Document Intelligencecited by 4
  15. 2026Infinity-Parser2 Technical Reportcited by 4
  16. 2024Transformers Utilization in Chart Understanding: A Review of Recent Advances & Future Trendscited by 3
  17. 2025Adaptive Markup Language Generation for Contextually-Grounded Visual Document Understandingcited by 3
  18. 2025In-Depth and In-Breadth: Pre-training Multimodal Language Models Customized for Comprehensive Chart Understandingcited by 3
  19. 2026Boosting Document Parsing Efficiency and Performance with Coarse-to-Fine Visual Processingcited by 3
  20. 2026ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formatscited by 2
  21. 2026Making Multimodal LLMs Reliable Chart Data Extractors: A Benchmark and Training Frameworkcited by 2
  22. 2026Self-Ensembling Vision-Language Models for Chart Data Extractioncited by 1
  23. 2025CycleChart: A Unified Consistency-Based Learning Framework for Bidirectional Chart Understanding and Generationcited by 0
  24. 2026CharTool: Tool-Integrated Visual Reasoning for Chart Understandingcited by 0
  25. 2026EpiCurveBench: Evaluating VLMs on Epidemic Curve Digitizationcited by 0
  26. 2026Code as Representation: A Compilable Parsing Paradigm for Academic Documentscited by 0
  27. 2026Chart2SVG: Editable SVG Generation from Raster Chart Imagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024AskChart: Universal Chart Understanding through Textual Enhancement
  2. 2026ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding
  3. 2025ChartAnchor: Chart Grounding with Structural-Semantic Fidelity
  4. 2026ChartZero: Synthetic Priors Enable Zero Shot Chart Data Extraction
  5. 2024ChartAdapter: Large Vision-Language Model for Chart Summarization
  6. 2023ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules
  7. 2024AltChart: Enhancing VLM-based Chart Summarization Through Multi-Pretext Tasks
  8. 2025START: Spatial and Textual Learning for Chart Understanding