# Molecular Structure Extraction From Documents Using Deep Learning

arXiv: https://arxiv.org/abs/1802.04903 · HTML: https://mlsanity.com/paper/1802.04903/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2018.

## Built on by (most cited first, 25 of 25)

- [ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area](https://mlsanity.com/paper/2408.07246/) (2024)
- [MolScribe: Robust Molecular Structure Recognition with Image-To-Graph Generation](https://mlsanity.com/paper/2205.14311/) (2022)
- [Artificial Intelligence in Drug Discovery: Applications and Techniques](https://mlsanity.com/paper/2106.05386/) (2021)
- [MolGrapher: Graph-based Visual Recognition of Chemical Structures](https://mlsanity.com/paper/2308.12234/) (2023)
- [RxnScribe: A Sequence Generation Model for Reaction Diagram Parsing](https://mlsanity.com/paper/2305.11845/) (2023)
- [OpenChemIE: An Information Extraction Toolkit For Chemistry Literature](https://mlsanity.com/paper/2404.01462/) (2024)
- [MolParser: End-to-end Visual Recognition of Molecule Structures in the Wild](https://mlsanity.com/paper/2411.11098/) (2024)
- [MolNexTR: A Generalized Deep Learning Model for Molecular Image Recognition](https://mlsanity.com/paper/2403.03691/) (2024)
- [GTR-CoT: Graph Traversal as Visual Chain of Thought for Molecular Structure Recognition](https://mlsanity.com/paper/2506.07553/) (2025)
- [Image-to-Graph Transformers for Chemical Structure Recognition](https://mlsanity.com/paper/2202.09580/) (2022)
- [End-to-End Attention-based Image Captioning](https://mlsanity.com/paper/2104.14721/) (2021)
- [Towards Large-scale Chemical Reaction Image Parsing via a Multimodal Large Language Model](https://mlsanity.com/paper/2503.08156/) (2025)
- [Self-Labeling of Fully Mediating Representations by Graph Alignment](https://mlsanity.com/paper/2103.14133/) (2021)
- [MolSight: Optical Chemical Structure Recognition with SMILES Pretraining, Multi-Granularity Learning and Reinforcement Learning](https://mlsanity.com/paper/2511.17300/) (2025)
- [RFL: Simplifying Chemical Structure Recognition with Ring-Free Language](https://mlsanity.com/paper/2412.07594/) (2024)
- [Fine-tuning DeepSeek-OCR-2 for Molecular Structure Recognition](https://mlsanity.com/paper/2604.03476/) (2026)
- [MolRecBench-Wild: A Real-World Benchmark for Optical Chemical Structure Recognition](https://mlsanity.com/paper/2605.05832/) (2026)
- [Real Data Closes Synthetic-to-Real Gap in Optical Chemical Structure Recognition](https://mlsanity.com/paper/2608.09100/) (2026)
- [ChemScraper: Leveraging PDF Graphics Instructions for Molecular Diagram Parsing](https://mlsanity.com/paper/2311.12161/) (2023)
- [zERExtractor:An Automated Platform for Enzyme-Catalyzed Reaction Data Extraction from Scientific Literature](https://mlsanity.com/paper/2508.09995/) (2025)
- [Building Deep Graph Predictors with Graph Imitation Learning](https://mlsanity.com/paper/2601.15133/) (2026)
- [COMO: Closed-Loop Optical Molecule Recognition with Minimum Risk Training](https://mlsanity.com/paper/2604.23546/) (2026)
- [ChemVA: Advancing Large Language Models on Chemical Reaction Diagrams Understanding](https://mlsanity.com/paper/2605.17214/) (2026)
- [VERDICT: Agreement Beats Pixel-Space Verification in Real-Document OCSR](https://mlsanity.com/paper/2608.22183/) (2026)
- [ViCoR: Reliable Molecular Structure Extraction via Spatially Aligned Verification and Executable Revision](https://mlsanity.com/paper/2609.33603/) (2026)

## Abstract (arXiv)

Chemical structure extraction from documents remains a hard problem due to both false positive identification of structures during segmentation and errors in the predicted structures. Current approaches rely on handcrafted rules and subroutines that perform reasonably well generally, but still routinely encounter situations where recognition rates are not yet satisfactory and systematic improvement is challenging. Complications impacting performance of current approaches include the diversity in visual styles used by various software to render structures, the frequent use of ad hoc annotations, and other challenges related to image quality, including resolution and noise. We here present end-to-end deep learning solutions for both segmenting molecular structures from documents and for predicting chemical structures from these segmented images. This deep learning-based approach does not require any handcrafted features, is learned directly from data, and is robust against variations in image quality and style. Using the deep-learning approach described herein we show that it is possible to perform well on both segmentation and prediction of low resolution images containing moderately sized molecules found in journal articles and patents.