# A General Framework for the Recognition of Online Handwritten Graphics

arXiv: https://arxiv.org/abs/1709.06389 · HTML: https://mlsanity.com/paper/1709.06389/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2017.

## Built on by (most cited first, 4 of 4)

- [Flowmind2Digital: The First Comprehensive Flowmind Recognition and Conversion Approach](https://mlsanity.com/paper/2401.03742/) (2024)
- [Automated Assessment of Multimodal Answer Sheets in the STEM domain](https://mlsanity.com/paper/2409.15749/) (2024)
- [Model-based inexact graph matching on top of CNNs for semantic scene understanding](https://mlsanity.com/paper/2301.07468/) (2023)
- [Attention Guidance Mechanism for Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2403.01756/) (2024)

## Abstract (arXiv)

We propose a new framework for the recognition of online handwritten graphics. Three main features of the framework are its ability to treat symbol and structural level information in an integrated way, its flexibility with respect to different families of graphics, and means to control the tradeoff between recognition effectiveness and computational cost. We model a graphic as a labeled graph generated from a graph grammar. Non-terminal vertices represent subcomponents, terminal vertices represent symbols, and edges represent relations between subcomponents or symbols. We then model the recognition problem as a graph parsing problem: given an input stroke set, we search for a parse tree that represents the best interpretation of the input. Our graph parsing algorithm generates multiple interpretations (consistent with the grammar) and then we extract an optimal interpretation according to a cost function that takes into consideration the likelihood scores of symbols and structures. The parsing algorithm consists in recursively partitioning the stroke set according to structures defined in the grammar and it does not impose constraints present in some previous works (e.g. stroke ordering). By avoiding such constraints and thanks to the powerful representativeness of graphs, our approach can be adapted to the recognition of different graphic notations. We show applications to the recognition of mathematical expressions and flowcharts. Experimentation shows that our method obtains state-of-the-art accuracy in both applications.