mlsanityData to 14 Oct 2026

MolScribe: Robust Molecular Structure Recognition with Image-To-Graph Generation

arXiv 2205.14311 ↗First posted 28 May 2022cs.CV cs.AIcited by 49

Not on the map. Why?

Abstract (arXiv)

Molecular structure recognition is the task of translating a molecular image into its graph structure. Significant variation in drawing styles and conventions exhibited in chemical literature poses a significant challenge for automating this task. In this paper, we propose MolScribe, a novel image-to-graph generation model that explicitly predicts atoms and bonds, along with their geometric layouts, to construct the molecular structure. Our model flexibly incorporates symbolic chemistry constraints to recognize chirality and expand abbreviated structures. We further develop data augmentation strategies to enhance the model robustness against domain shifts. In experiments on both synthetic and realistic molecular images, MolScribe significantly outperforms previous models, achieving 76-93% accuracy on public benchmarks. Chemists can also easily verify MolScribe's prediction, informed by its confidence estimation and atom-level alignment with the input image. MolScribe is publicly available through Python and web interfaces: https://github.com/thomas0809/MolScribe.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). Builds on 8 of its 38 references that are in the map. First posted on arXiv 28 May 2022.

Builds on

8 of 38 references.

References · 8

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2018Relational inductive biases, deep learning, and graph networkscited by 1,360
  4. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 349
  5. 2018Molecular Structure Extraction From Documents Using Deep Learningcited by 25
  6. 2020ChemGrapher: Optical Graph Recognition of Chemical Compounds by Deep Learningcited by 18
  7. 2022MolMiner: You only look once for chemical structure recognitioncited by 10
  8. 2022Image-to-Graph Transformers for Chemical Structure Recognitioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 10; 2025: 16; 2026: 20.

49 papers.

Sort:

By area:

  1. 2024ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Areacited by 71
  2. 2025SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversioncited by 53
  3. 2023RxnScribe: A Sequence Generation Model for Reaction Diagram Parsingcited by 17
  4. 2024OpenChemIE: An Information Extraction Toolkit For Chemistry Literaturecited by 17
  5. 2024MolParser: End-to-end Visual Recognition of Molecule Structures in the Wildcited by 17
  6. 2025ChemMLLM: Chemical Multimodal Large Language Modelcited by 17
  7. 2024MolNexTR: A Generalized Deep Learning Model for Molecular Image Recognitioncited by 16
  8. 2024ChemDFM-X: Towards Large Multimodal Model for Chemistrycited by 16
  9. 2023Predictive Chemistry Augmented with Text Retrievalcited by 10
  10. 2025OCSU: Optical Chemical Structure Understanding for Molecule-centric Scientific Discoverycited by 8
  11. 2025A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Toolscited by 7
  12. 2025TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Taskscited by 6
37 more
  1. 2025Evaluating Large Language Models on Multimodal Chemistry Olympiad Examscited by 6
  2. 2024Atom-Level Optical Chemical Structure Recognition with Limited Supervisioncited by 5
  3. 2025MarkushGrapher: Joint Visual and Textual Recognition of Markush Structurescited by 5
  4. 2024textttPatentAgent: Intelligent Agent for Automated Pharmaceutical Patent Analysiscited by 4
  5. 2025Towards Large-scale Chemical Reaction Image Parsing via a Multimodal Large Language Modelcited by 4
  6. 2025RxnCaption: Reformulating Reaction Diagram Parsing as Visual Prompt Guided Captioningcited by 3
  7. 2025ChemLabs on ChemO: A Multi-Agent System for Multimodal Reasoning on IChO 2025cited by 3
  8. 2025Uni-Parser Technical Reportcited by 3
  9. 2024SMiCRM: A Benchmark Dataset of Mechanistic Molecular Imagescited by 2
  10. 2025MolMole: Molecule Mining from Scientific Literaturecited by 2
  11. 2025Doc2SAR: A Synergistic Framework for High-Fidelity Extraction of Structure-Activity Relationships from Scientific Documentscited by 2
  12. 2025MolSight: Optical Chemical Structure Recognition with SMILES Pretraining, Multi-Granularity Learning and Reinforcement Learningcited by 2
  13. 2026MarkushGrapher-2: End-to-end Multimodal Recognition of Chemical Structurescited by 2
  14. 2026ChemVLR: Prioritizing Reasoning in Perception for Chemical Vision-Language Understandingcited by 2
  15. 2024Accelerating materials discovery for polymer solar cells: Data-driven insights enabled by natural language processingcited by 1
  16. 2024RFL: Simplifying Chemical Structure Recognition with Ring-Free Languagecited by 1
  17. 2026Fine-tuning DeepSeek-OCR-2 for Molecular Structure Recognitioncited by 1
  18. 2026BioMiner: A Multi-modal System for Automated Mining of Protein-Ligand Bioactivity Data from Literaturecited by 1
  19. 2026MolRecBench-Wild: A Real-World Benchmark for Optical Chemical Structure Recognitioncited by 1
  20. 2026MarkushGlyph and OCSRGlyph: Improved Chemical Structure Recognitioncited by 1
  21. 2026MinerU.Chem: A High-Precision System for Optical Chemical Structure and Reaction Recognitioncited by 1
  22. 2026Real Data Closes Synthetic-to-Real Gap in Optical Chemical Structure Recognitioncited by 1
  23. 2023ChemScraper: Leveraging PDF Graphics Instructions for Molecular Diagram Parsingcited by 0
  24. 2025SubGrapher: Visual Fingerprinting of Chemical Structurescited by 0
  25. 2025zERExtractor:An Automated Platform for Enzyme-Catalyzed Reaction Data Extraction from Scientific Literaturecited by 0
  26. 2026DeepMoLM: Leveraging Visual and Geometric Structural Information for Molecule-Text Modelingcited by 0
  27. 2026Building Deep Graph Predictors with Graph Imitation Learningcited by 0
  28. 2026COMO: Closed-Loop Optical Molecule Recognition with Minimum Risk Trainingcited by 0
  29. 2026ChemVA: Advancing Large Language Models on Chemical Reaction Diagrams Understandingcited by 0
  30. 2026MolSight: A Graph-Aware Vision-Language Model for Unified Chemical Image Understandingcited by 0
  31. 2026VERDICT: Agreement Beats Pixel-Space Verification in Real-Document OCSRcited by 0
  32. 2026Molecular LLM Agents: From Architectural Design to Scientific Autonomycited by 0
  33. 2026Differential Learning for Robust Prediction of Thermal Stability with Application to Energetic Materialscited by 0
  34. 2026MolParser-Mobile: Ultrafast OCSR System for Large-Scale Chemical Literature Miningcited by 0
  35. 2026ViCoR: Reliable Molecular Structure Extraction via Spatially Aligned Verification and Executable Revisioncited by 0
  36. 2026Learning Transferable Reaction Mechanisms from Visual Chemical Knowledgecited by 0
  37. 2026ChemExplain: A Multimodal Benchmark for Explanatory Chemical Reasoningcited by 0