# EpiCurveBench: Evaluating VLMs on Epidemic Curve Digitization

arXiv: https://arxiv.org/abs/2605.27195 · HTML: https://mlsanity.com/paper/2605.27195/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 26 May 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 10 of 10)

- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [PlotQA: Reasoning over Scientific Plots](https://mlsanity.com/paper/1909.00997/) (2019)
- [DePlot: One-shot visual language reasoning by plot-to-table translation](https://mlsanity.com/paper/2212.10505/) (2022)
- [MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering](https://mlsanity.com/paper/2212.09662/) (2022)
- [ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering](https://mlsanity.com/paper/2504.05506/) (2025)
- [TinyChart: Efficient Chart Understanding with Visual Token Merging and Program-of-Thoughts Learning](https://mlsanity.com/paper/2404.16635/) (2024)
- [ICDAR 2019 Competition on Scene Text Visual Question Answering](https://mlsanity.com/paper/1907.00490/) (2019)
- [OneChart: Purify the Chart Structural Extraction via One Auxiliary Token](https://mlsanity.com/paper/2404.09987/) (2024)
- [StructChart: On the Schema, Metric, and Augmentation for Visual Chart Understanding](https://mlsanity.com/paper/2309.11268/) (2023)

## Abstract (arXiv)

Chart-to-data extraction with vision-language models (VLMs) is increasingly evaluated on benchmarks that show diminishing headroom (frontier VLMs exceed 89% on ChartQA) and with metrics that treat extracted points as unordered key-value pairs, ignoring the temporal structure of time series and penalizing small alignment shifts as catastrophic failures. We address both gaps with EpiCurveBench, a benchmark of 1,000 real-world epidemic curve images curated from diverse public-health sources, and EpiCurveSimilarity (ECS), an evaluation metric that aligns predicted and ground-truth series via dynamic programming, tolerating local temporal shifts and gaps while penalizing them proportionally. Evaluating six methods--three frontier closed VLMs, one open VLM, and two specialized chart-extraction systems--we find the strongest model reaches only 52.3% ECS, and that ECS spreads the four general-purpose VLMs over a 25-point range where key-value metrics (RMS, SCRM) compress them into a 5-point band. We further validate ECS against four downstream epidemiological summary statistics, finding that higher ECS predicts smaller errors in total counts, peak timing, and peak magnitude, and higher growth-rate fidelity; across all four, ECS correlates 1.5--3.6 times more strongly than Dynamic Time Warping, which lacks a gap penalty and therefore cannot distinguish a truncated prediction from a temporally faithful one. EpiCurveBench targets a high-impact public-health application--unlocking decades of outbreak data trapped in published figures--but the benchmark and metric apply directly to any structured time-series chart-extraction setting.