# ChartMark: A Structured Grammar for Chart Annotation

arXiv: https://arxiv.org/abs/2507.21810 · HTML: https://mlsanity.com/paper/2507.21810/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 43 references that are in the map.
- First posted on arXiv 29 Jul 2025.

## Builds on (most cited first, 15 of 15)

- [Towards Natural Language Interfaces for Data Visualization: A Survey](https://mlsanity.com/paper/2109.03506/) (2021)
- [ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answering](https://mlsanity.com/paper/2405.07001/) (2024)
- [HAIChart: Human and AI Paired Visualization System](https://mlsanity.com/paper/2406.11033/) (2024)
- [Effortless Data Exploration with zenvisage: An Expressive and Interactive Visual Analytics System](https://mlsanity.com/paper/1604.03583/) (2016)
- [AskChart: Universal Chart Understanding through Textual Enhancement](https://mlsanity.com/paper/2412.19146/) (2024)
- [Reflecting on Design Paradigms of Animated Data Video Tools](https://mlsanity.com/paper/2502.04801/) (2025)
- [Visual Data Analysis with Task-based Recommendations](https://mlsanity.com/paper/2205.03183/) (2022)
- [Debugging Defective Visualizations: Empirical Insights Informing a Human-AI Co-Debugging System](https://mlsanity.com/paper/2412.07673/) (2024)
- [ChartCards: A Chart-Metadata Generation Framework for Multi-Task Chart Understanding](https://mlsanity.com/paper/2505.15046/) (2025)
- [A Qualitative Analysis of Common Practices in Annotations: A Taxonomy and Design Space](https://mlsanity.com/paper/2306.06043/) (2023)
- [Prompting Generative AI with Interaction-Augmented Instructions](https://mlsanity.com/paper/2503.02874/) (2025)
- [Graph Exploration with Embedding-Guided Layouts](https://mlsanity.com/paper/2208.13699/) (2022)
- [VizBelle: A Design Space of Embellishments for Data Visualization](https://mlsanity.com/paper/2209.03642/) (2022)
- [PyGWalker: On-the-fly Assistant for Exploratory Visual Data Analysis](https://mlsanity.com/paper/2406.11637/) (2024)
- [FinFlier: Automating Graphical Overlays for Financial Visualizations with Knowledge-Grounding Large Language Model](https://mlsanity.com/paper/2412.06821/) (2024)

## Built on by (most cited first, 6 of 6)

- [DataSpace: Benchmarking Data Agents for Verifiable Analytics over Heterogeneous Workspaces](https://mlsanity.com/paper/2608.03451/) (2026)
- [DataMagic: Transforming Tabular Data into Data Insight Video](https://mlsanity.com/paper/2606.20388/) (2026)
- [AnnoBench: A Benchmark for Visualization Annotation Generation](https://mlsanity.com/paper/2607.25911/) (2026)
- [ChartAnno: Benchmarking Multimodal Large Language Models for Chart Annotation Generation](https://mlsanity.com/paper/2608.03464/) (2026)
- [FinReportBench: Measuring and Improving Institution-Grade Financial Report Generation](https://mlsanity.com/paper/2608.04374/) (2026)
- [Where LLMs Fail with Visualization DSLs](https://mlsanity.com/paper/2610.01873/) (2026)

## Abstract (arXiv)

Chart annotations enhance visualization accessibility but suffer from fragmented, non-standardized representations that limit cross-platform reuse. We propose ChartMark, a structured grammar that separates annotation semantics from visualization implementations. ChartMark features a hierarchical framework mapping onto annotation dimensions (e.g., task, chart context), supporting both abstract intents and precise visual details. Our toolkit demonstrates converting ChartMark specifications into Vega-Lite visualizations, highlighting its flexibility, expressiveness, and practical applicability.