# MolScribe: Robust Molecular Structure Recognition with Image-To-Graph Generation

arXiv: https://arxiv.org/abs/2205.14311 · HTML: https://mlsanity.com/paper/2205.14311/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 38 references that are in the map.
- First posted on arXiv 28 May 2022.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Pix2seq: A Language Modeling Framework for Object Detection](https://mlsanity.com/paper/2109.10852/) (2021)
- [Molecular Structure Extraction From Documents Using Deep Learning](https://mlsanity.com/paper/1802.04903/) (2018)
- [ChemGrapher: Optical Graph Recognition of Chemical Compounds by Deep Learning](https://mlsanity.com/paper/2002.09914/) (2020)
- [MolMiner: You only look once for chemical structure recognition](https://mlsanity.com/paper/2205.11016/) (2022)
- [Image-to-Graph Transformers for Chemical Structure Recognition](https://mlsanity.com/paper/2202.09580/) (2022)

## Built on by (most cited first, 30 of 49)

- [ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area](https://mlsanity.com/paper/2408.07246/) (2024)
- [SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion](https://mlsanity.com/paper/2503.11576/) (2025)
- [RxnScribe: A Sequence Generation Model for Reaction Diagram Parsing](https://mlsanity.com/paper/2305.11845/) (2023)
- [OpenChemIE: An Information Extraction Toolkit For Chemistry Literature](https://mlsanity.com/paper/2404.01462/) (2024)
- [MolParser: End-to-end Visual Recognition of Molecule Structures in the Wild](https://mlsanity.com/paper/2411.11098/) (2024)
- [ChemMLLM: Chemical Multimodal Large Language Model](https://mlsanity.com/paper/2505.16326/) (2025)
- [MolNexTR: A Generalized Deep Learning Model for Molecular Image Recognition](https://mlsanity.com/paper/2403.03691/) (2024)
- [ChemDFM-X: Towards Large Multimodal Model for Chemistry](https://mlsanity.com/paper/2409.13194/) (2024)
- [Predictive Chemistry Augmented with Text Retrieval](https://mlsanity.com/paper/2312.04881/) (2023)
- [OCSU: Optical Chemical Structure Understanding for Molecule-centric Scientific Discovery](https://mlsanity.com/paper/2501.15415/) (2025)
- [A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Tools](https://mlsanity.com/paper/2506.20743/) (2025)
- [TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Tasks](https://mlsanity.com/paper/2511.06283/) (2025)
- [Evaluating Large Language Models on Multimodal Chemistry Olympiad Exams](https://mlsanity.com/paper/2512.14989/) (2025)
- [Atom-Level Optical Chemical Structure Recognition with Limited Supervision](https://mlsanity.com/paper/2404.01743/) (2024)
- [MarkushGrapher: Joint Visual and Textual Recognition of Markush Structures](https://mlsanity.com/paper/2503.16096/) (2025)
- [textttPatentAgent: Intelligent Agent for Automated Pharmaceutical Patent Analysis](https://mlsanity.com/paper/2410.21312/) (2024)
- [Towards Large-scale Chemical Reaction Image Parsing via a Multimodal Large Language Model](https://mlsanity.com/paper/2503.08156/) (2025)
- [RxnCaption: Reformulating Reaction Diagram Parsing as Visual Prompt Guided Captioning](https://mlsanity.com/paper/2511.02384/) (2025)
- [ChemLabs on ChemO: A Multi-Agent System for Multimodal Reasoning on IChO 2025](https://mlsanity.com/paper/2511.16205/) (2025)
- [Uni-Parser Technical Report](https://mlsanity.com/paper/2512.15098/) (2025)
- [SMiCRM: A Benchmark Dataset of Mechanistic Molecular Images](https://mlsanity.com/paper/2407.18338/) (2024)
- [MolMole: Molecule Mining from Scientific Literature](https://mlsanity.com/paper/2505.03777/) (2025)
- [Doc2SAR: A Synergistic Framework for High-Fidelity Extraction of Structure-Activity Relationships from Scientific Documents](https://mlsanity.com/paper/2506.21625/) (2025)
- [MolSight: Optical Chemical Structure Recognition with SMILES Pretraining, Multi-Granularity Learning and Reinforcement Learning](https://mlsanity.com/paper/2511.17300/) (2025)
- [MarkushGrapher-2: End-to-end Multimodal Recognition of Chemical Structures](https://mlsanity.com/paper/2603.28550/) (2026)
- [ChemVLR: Prioritizing Reasoning in Perception for Chemical Vision-Language Understanding](https://mlsanity.com/paper/2604.06685/) (2026)
- [Accelerating materials discovery for polymer solar cells: Data-driven insights enabled by natural language processing](https://mlsanity.com/paper/2402.19462/) (2024)
- [RFL: Simplifying Chemical Structure Recognition with Ring-Free Language](https://mlsanity.com/paper/2412.07594/) (2024)
- [Fine-tuning DeepSeek-OCR-2 for Molecular Structure Recognition](https://mlsanity.com/paper/2604.03476/) (2026)
- [BioMiner: A Multi-modal System for Automated Mining of Protein-Ligand Bioactivity Data from Literature](https://mlsanity.com/paper/2604.21508/) (2026)

## Abstract (arXiv)

Molecular structure recognition is the task of translating a molecular image into its graph structure. Significant variation in drawing styles and conventions exhibited in chemical literature poses a significant challenge for automating this task. In this paper, we propose MolScribe, a novel image-to-graph generation model that explicitly predicts atoms and bonds, along with their geometric layouts, to construct the molecular structure. Our model flexibly incorporates symbolic chemistry constraints to recognize chirality and expand abbreviated structures. We further develop data augmentation strategies to enhance the model robustness against domain shifts. In experiments on both synthetic and realistic molecular images, MolScribe significantly outperforms previous models, achieving 76-93% accuracy on public benchmarks. Chemists can also easily verify MolScribe's prediction, informed by its confidence estimation and atom-level alignment with the input image. MolScribe is publicly available through Python and web interfaces: https://github.com/thomas0809/MolScribe.