PA-CDM: Position-Aware Character Detection Matching for Evaluating Handwritten Mathematical Expression Recognition
Not on the map. Why?
Abstract (arXiv)
Handwritten mathematical expression recognition (HMER) is conventionally scored by exact-match rates and string-similarity metrics that are blind to where an error occurs: two predictions with identical token-error counts receive identical scores whether they misplace a subscript or swap the operands of a fraction. Render-based character detection matching (CDM) aligns glyphs robustly but remains position-blind---on controlled fraction-operand swaps it scores 0.8595 where position-aware scoring yields 0.6253. Tree-edit metrics exhibit a complementary blind spot: rewrites outside the parser's normalization coverage are penalized as structural errors (0.8552 where render-based metrics score 1.0). We propose PA-CDM, a position-aware metric that couples character detection matching with position-forest encoding and divergence-level weighting; StructPerturb v2.0, a frozen benchmark of 1,340 controlled perturbation pairs across 15 type--intensity cells; and a cross-metric consistency protocol combining a sensitivity matrix, a human study, and LLM-judge calibration. In a six-annotator study, PA-CDM attains the highest correlation with human judgments among seven automatic metrics (Spearman rho=0.9535, n=990). A frontier LLM judge correlates slightly higher (rho=0.9613) but is costly, nondeterministic, and API-dependent; PA-CDM approaches it at zero marginal cost with deterministic, diagnosable behavior.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 22 references that are in the map. First posted on arXiv 11 Sep 2026.
Builds on
13 of 22 references.
References · 13
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2024A Survey on LLM-as-a-Judgecited by 1,423
- 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotationscited by 150
- 2024UniMERNet: A Universal Network for Real-World Mathematical Expression Recognitioncited by 42
- 2024Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matchingcited by 41
- 2022When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognitioncited by 35
- 2021Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformercited by 34
- 2022CoMER: Modeling Coverage for Transformer-based Handwritten Mathematical Expression Recognitioncited by 24
- 2024MathWriting: A Dataset For Handwritten Mathematical Expression Recognitioncited by 21
- 2024PosFormer: Recognizing Complex Handwritten Mathematical Expression with Position Forest Transformercited by 18
- 2025VEHME: A Vision-Language Model For Evaluating Handwritten Mathematics Expressionscited by 4
Built on by
Not cited yet.