# Image-to-Graph Transformers for Chemical Structure Recognition

arXiv: https://arxiv.org/abs/2202.09580 · HTML: https://mlsanity.com/paper/2202.09580/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 16 references that are in the map.
- First posted on arXiv 19 Feb 2022.

## Builds on (most cited first, 5 of 5)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [MolGAN: An implicit generative model for small molecular graphs](https://mlsanity.com/paper/1805.11973/) (2018)
- [Molecular Structure Extraction From Documents Using Deep Learning](https://mlsanity.com/paper/1802.04903/) (2018)
- [ChemGrapher: Optical Graph Recognition of Chemical Compounds by Deep Learning](https://mlsanity.com/paper/2002.09914/) (2020)
- [Graph-Aware Transformer: Is Attention All Graphs Need?](https://mlsanity.com/paper/2006.05213/) (2020)

## Built on by (most cited first, 6 of 6)

- [MolScribe: Robust Molecular Structure Recognition with Image-To-Graph Generation](https://mlsanity.com/paper/2205.14311/) (2022)
- [MolGrapher: Graph-based Visual Recognition of Chemical Structures](https://mlsanity.com/paper/2308.12234/) (2023)
- [MolParser: End-to-end Visual Recognition of Molecule Structures in the Wild](https://mlsanity.com/paper/2411.11098/) (2024)
- [MolNexTR: A Generalized Deep Learning Model for Molecular Image Recognition](https://mlsanity.com/paper/2403.03691/) (2024)
- [ChemScraper: Leveraging PDF Graphics Instructions for Molecular Diagram Parsing](https://mlsanity.com/paper/2311.12161/) (2023)
- [zERExtractor:An Automated Platform for Enzyme-Catalyzed Reaction Data Extraction from Scientific Literature](https://mlsanity.com/paper/2508.09995/) (2025)

## Abstract (arXiv)

For several decades, chemical knowledge has been published in written text, and there have been many attempts to make it accessible, for example, by transforming such natural language text to a structured format. Although the discovered chemical itself commonly represented in an image is the most important part, the correct recognition of the molecular structure from the image in literature still remains a hard problem since they are often abbreviated to reduce the complexity and drawn in many different styles. In this paper, we present a deep learning model to extract molecular structures from images. The proposed model is designed to transform the molecular image directly into the corresponding graph, which makes it capable of handling non-atomic symbols for abbreviations. Also, by end-to-end learning approach it can fully utilize many open image-molecule pair data from various sources, and hence it is more robust to image style variation than other tools. The experimental results show that the proposed model outperforms the existing models with 17.1 % and 12.8 % relative improvement for well-known benchmark datasets and large molecular images that we collected from literature, respectively.