# GTR-CoT: Graph Traversal as Visual Chain of Thought for Molecular Structure Recognition

arXiv: https://arxiv.org/abs/2506.07553 · HTML: https://mlsanity.com/paper/2506.07553/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 18 of its 55 references that are in the map.
- First posted on arXiv 9 Jun 2025.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)

## Builds on (most cited first, 18 of 18)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities](https://mlsanity.com/paper/2308.02490/) (2023)
- [MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark](https://mlsanity.com/paper/2409.02813/) (2024)
- [General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model](https://mlsanity.com/paper/2409.01704/) (2024)
- [LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model](https://mlsanity.com/paper/2402.02544/) (2024)
- [VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysis](https://mlsanity.com/paper/2403.20213/) (2024)
- [ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area](https://mlsanity.com/paper/2408.07246/) (2024)
- [CityLLaVA: Efficient Fine-Tuning for VLMs in City Scenario](https://mlsanity.com/paper/2405.03194/) (2024)
- [Molecular Structure Extraction From Documents Using Deep Learning](https://mlsanity.com/paper/1802.04903/) (2018)
- [MolParser: End-to-end Visual Recognition of Molecule Structures in the Wild](https://mlsanity.com/paper/2411.11098/) (2024)
- [OCSU: Optical Chemical Structure Understanding for Molecule-centric Scientific Discovery](https://mlsanity.com/paper/2501.15415/) (2025)
- [IMG2SMI: Translating Molecular Structure Images to Simplified Molecular-input Line-entry System](https://mlsanity.com/paper/2109.04202/) (2021)
- [Atom-Level Optical Chemical Structure Recognition with Limited Supervision](https://mlsanity.com/paper/2404.01743/) (2024)
- [MarkushGrapher: Joint Visual and Textual Recognition of Markush Structures](https://mlsanity.com/paper/2503.16096/) (2025)
- [Benchmarking and Boosting Multilingual Capabilities of LVLMs via OCR-Centric Reinforcement Learning](https://mlsanity.com/paper/2503.18484/) (2025)

## Built on by (most cited first, 8 of 8)

- [RxnCaption: Reformulating Reaction Diagram Parsing as Visual Prompt Guided Captioning](https://mlsanity.com/paper/2511.02384/) (2025)
- [Fine-tuning DeepSeek-OCR-2 for Molecular Structure Recognition](https://mlsanity.com/paper/2604.03476/) (2026)
- [MolRecBench-Wild: A Real-World Benchmark for Optical Chemical Structure Recognition](https://mlsanity.com/paper/2605.05832/) (2026)
- [MinerU.Chem: A High-Precision System for Optical Chemical Structure and Reaction Recognition](https://mlsanity.com/paper/2608.03525/) (2026)
- [MarsRetrieval: Benchmarking Vision-Language Models for Planetary-Scale Geospatial Retrieval on Mars](https://mlsanity.com/paper/2602.13961/) (2026)
- [ChemVA: Advancing Large Language Models on Chemical Reaction Diagrams Understanding](https://mlsanity.com/paper/2605.17214/) (2026)
- [Localize, Then Reason: Visual Latent Structural Reasoning for Molecular Properties and Edits](https://mlsanity.com/paper/2608.13244/) (2026)
- [RxnOptBench: Benchmarking LLMs for Reaction-Condition Optimization in Organic Methodology](https://mlsanity.com/paper/2610.02242/) (2026)

## Abstract (arXiv)

Optical Chemical Structure Recognition (OCSR) is essential for converting molecular images into machine-readable formats. While recent vision-language models (VLMs) have shown promise, their image-captioning approach often struggles with complex molecular structures and inconsistent annotations. To address these issues, we introduce GTR-VL, featuring two key innovations: (1) the textitGraph Traversal as Visual Chain of Thought mechanism that emulates human reasoning by incrementally parsing molecular graphs through sequential atom-bond predictions, and (2) the data-centric textitFaithfully Recognize What You've Seen principle, which aligns abbreviated structures in images with their expanded annotations. For hand-drawn OCSR tasks, where datasets lack graph annotations and only provide final SMILES, we apply reinforcement learning using the GRPO method, introducing reward mechanisms like format reward, graph reward, and SMILES reward. This approach significantly enhances performance in hand-drawn recognition tasks through weak supervision. We developed GTR-1.3M, a large-scale instruction-tuning dataset with corrected annotations, and MolRec-Bench, the first benchmark for fine-grained evaluation of graph-parsing accuracy in OCSR. Our two-stage training scheme involves SFT training for printed images and the GRPO method for transferring capabilities to hand-drawn tasks. Experiments show that GTR-VL outperforms specialist models, chemistry-domain VLMs, and commercial VLMs on both printed and hand-drawn datasets.