# ChemGrapher: Optical Graph Recognition of Chemical Compounds by Deep Learning

arXiv: https://arxiv.org/abs/2002.09914 · HTML: https://mlsanity.com/paper/2002.09914/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2020.

## Built on by (most cited first, 18 of 18)

- [ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area](https://mlsanity.com/paper/2408.07246/) (2024)
- [MolScribe: Robust Molecular Structure Recognition with Image-To-Graph Generation](https://mlsanity.com/paper/2205.14311/) (2022)
- [MolGrapher: Graph-based Visual Recognition of Chemical Structures](https://mlsanity.com/paper/2308.12234/) (2023)
- [RxnScribe: A Sequence Generation Model for Reaction Diagram Parsing](https://mlsanity.com/paper/2305.11845/) (2023)
- [OpenChemIE: An Information Extraction Toolkit For Chemistry Literature](https://mlsanity.com/paper/2404.01462/) (2024)
- [MolParser: End-to-end Visual Recognition of Molecule Structures in the Wild](https://mlsanity.com/paper/2411.11098/) (2024)
- [MolNexTR: A Generalized Deep Learning Model for Molecular Image Recognition](https://mlsanity.com/paper/2403.03691/) (2024)
- [MolMiner: You only look once for chemical structure recognition](https://mlsanity.com/paper/2205.11016/) (2022)
- [IMG2SMI: Translating Molecular Structure Images to Simplified Molecular-input Line-entry System](https://mlsanity.com/paper/2109.04202/) (2021)
- [Image-to-Graph Transformers for Chemical Structure Recognition](https://mlsanity.com/paper/2202.09580/) (2022)
- [Atom-Level Optical Chemical Structure Recognition with Limited Supervision](https://mlsanity.com/paper/2404.01743/) (2024)
- [End-to-End Attention-based Image Captioning](https://mlsanity.com/paper/2104.14721/) (2021)
- [Towards Large-scale Chemical Reaction Image Parsing via a Multimodal Large Language Model](https://mlsanity.com/paper/2503.08156/) (2025)
- [Self-Labeling of Fully Mediating Representations by Graph Alignment](https://mlsanity.com/paper/2103.14133/) (2021)
- [Weakly Supervised Knowledge Transfer with Probabilistic Logical Reasoning for Object Detection](https://mlsanity.com/paper/2303.05148/) (2023)
- [RFL: Simplifying Chemical Structure Recognition with Ring-Free Language](https://mlsanity.com/paper/2412.07594/) (2024)
- [SubGrapher: Visual Fingerprinting of Chemical Structures](https://mlsanity.com/paper/2504.19695/) (2025)
- [A Comparative Study of QSPR Methods on a Unique Multitask PAMPA dataset](https://mlsanity.com/paper/2605.00508/) (2026)

## Abstract (arXiv)

In drug discovery, knowledge of the graph structure of chemical compounds is essential. Many thousands of scientific articles in chemistry and pharmaceutical sciences have investigated chemical compounds, but in cases the details of the structure of these chemical compounds is published only as an images. A tool to analyze these images automatically and convert them into a chemical graph structure would be useful for many applications, such drug discovery. A few such tools are available and they are mostly derived from optical character recognition. However, our evaluation of the performance of those tools reveals that they make often mistakes in detecting the correct bond multiplicity and stereochemical information. In addition, errors sometimes even lead to missing atoms in the resulting graph. In our work, we address these issues by developing a compound recognition method based on machine learning. More specifically, we develop a deep neural network model for optical compound recognition. The deep learning solution presented here consists of a segmentation model, followed by three classification models that predict atom locations, bonds and charges. Furthermore, this model not only predicts the graph structure of the molecule but also produces all information necessary to relate each component of the resulting graph to the source image. This solution is scalable and could rapidly process thousands of images. Finally, we compare empirically the proposed method to a well-established tool and observe significant error reductions.