# MolGrapher: Graph-based Visual Recognition of Chemical Structures

arXiv: https://arxiv.org/abs/2308.12234 · HTML: https://mlsanity.com/paper/2308.12234/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 41 references that are in the map.
- First posted on arXiv 23 Aug 2023.

## Builds on (most cited first, 10 of 10)

- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [PP-OCR: A Practical Ultra Lightweight OCR System](https://mlsanity.com/paper/2009.09941/) (2020)
- [SELFIES and the future of molecular string representations](https://mlsanity.com/paper/2204.00056/) (2022)
- [Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation](https://mlsanity.com/paper/2012.15175/) (2020)
- [Molecular Structure Extraction From Documents Using Deep Learning](https://mlsanity.com/paper/1802.04903/) (2018)
- [ChemGrapher: Optical Graph Recognition of Chemical Compounds by Deep Learning](https://mlsanity.com/paper/2002.09914/) (2020)
- [MolMiner: You only look once for chemical structure recognition](https://mlsanity.com/paper/2205.11016/) (2022)
- [IMG2SMI: Translating Molecular Structure Images to Simplified Molecular-input Line-entry System](https://mlsanity.com/paper/2109.04202/) (2021)
- [Image-to-Graph Transformers for Chemical Structure Recognition](https://mlsanity.com/paper/2202.09580/) (2022)
- [End-to-End Attention-based Image Captioning](https://mlsanity.com/paper/2104.14721/) (2021)

## Built on by (most cited first, 24 of 24)

- [SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion](https://mlsanity.com/paper/2503.11576/) (2025)
- [MolParser: End-to-end Visual Recognition of Molecule Structures in the Wild](https://mlsanity.com/paper/2411.11098/) (2024)
- [Benchmarking Multimodal LLMs on Recognition and Understanding over Chemical Tables](https://mlsanity.com/paper/2506.11375/) (2025)
- [OCSU: Optical Chemical Structure Understanding for Molecule-centric Scientific Discovery](https://mlsanity.com/paper/2501.15415/) (2025)
- [Advanced Layout Analysis Models for Docling](https://mlsanity.com/paper/2509.11720/) (2025)
- [TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Tasks](https://mlsanity.com/paper/2511.06283/) (2025)
- [MarkushGrapher: Joint Visual and Textual Recognition of Markush Structures](https://mlsanity.com/paper/2503.16096/) (2025)
- [MolMole: Molecule Mining from Scientific Literature](https://mlsanity.com/paper/2505.03777/) (2025)
- [Doc2SAR: A Synergistic Framework for High-Fidelity Extraction of Structure-Activity Relationships from Scientific Documents](https://mlsanity.com/paper/2506.21625/) (2025)
- [MolSight: Optical Chemical Structure Recognition with SMILES Pretraining, Multi-Granularity Learning and Reinforcement Learning](https://mlsanity.com/paper/2511.17300/) (2025)
- [MarkushGrapher-2: End-to-end Multimodal Recognition of Chemical Structures](https://mlsanity.com/paper/2603.28550/) (2026)
- [ChemVLR: Prioritizing Reasoning in Perception for Chemical Vision-Language Understanding](https://mlsanity.com/paper/2604.06685/) (2026)
- [RFL: Simplifying Chemical Structure Recognition with Ring-Free Language](https://mlsanity.com/paper/2412.07594/) (2024)
- [Large Language Models Transform Organic Synthesis From Reaction Prediction to Automation](https://mlsanity.com/paper/2508.05427/) (2025)
- [MolRecBench-Wild: A Real-World Benchmark for Optical Chemical Structure Recognition](https://mlsanity.com/paper/2605.05832/) (2026)
- [MarkushGlyph and OCSRGlyph: Improved Chemical Structure Recognition](https://mlsanity.com/paper/2607.28532/) (2026)
- [MinerU.Chem: A High-Precision System for Optical Chemical Structure and Reaction Recognition](https://mlsanity.com/paper/2608.03525/) (2026)
- [ChemScraper: Leveraging PDF Graphics Instructions for Molecular Diagram Parsing](https://mlsanity.com/paper/2311.12161/) (2023)
- [SubGrapher: Visual Fingerprinting of Chemical Structures](https://mlsanity.com/paper/2504.19695/) (2025)
- [Building Deep Graph Predictors with Graph Imitation Learning](https://mlsanity.com/paper/2601.15133/) (2026)
- [COMO: Closed-Loop Optical Molecule Recognition with Minimum Risk Training](https://mlsanity.com/paper/2604.23546/) (2026)
- [MolParser-Mobile: Ultrafast OCSR System for Large-Scale Chemical Literature Mining](https://mlsanity.com/paper/2609.05807/) (2026)
- [ViCoR: Reliable Molecular Structure Extraction via Spatially Aligned Verification and Executable Revision](https://mlsanity.com/paper/2609.33603/) (2026)
- [R-GroundBench: A Diagnostic Benchmark for R-Group Groundingin Markush Molecular Editing](https://mlsanity.com/paper/2610.00700/) (2026)

## Abstract (arXiv)

The automatic analysis of chemical literature has immense potential to accelerate the discovery of new materials and drugs. Much of the critical information in patent documents and scientific articles is contained in figures, depicting the molecule structures. However, automatically parsing the exact chemical structure is a formidable challenge, due to the amount of detailed information, the diversity of drawing styles, and the need for training data. In this work, we introduce MolGrapher to recognize chemical structures visually. First, a deep keypoint detector detects the atoms. Second, we treat all candidate atoms and bonds as nodes and put them in a graph. This construct allows a natural graph representation of the molecule. Last, we classify atom and bond nodes in the graph with a Graph Neural Network. To address the lack of real training data, we propose a synthetic data generation pipeline producing diverse and realistic results. In addition, we introduce a large-scale benchmark of annotated real molecule images, USPTO-30K, to spur research on this critical topic. Extensive experiments on five datasets show that our approach significantly outperforms classical and learning-based methods in most settings. Code, models, and datasets are available.