Molecular Identifier Visual Prompt and Verifiable Reinforcement Learning for Chemical Reaction Diagram Parsing
On the map: Deep reinforcement learning methods → RL for energy, networks and operations
Abstract (arXiv)
Reaction diagram parsing (RxnDP) is critical for extracting chemical synthesis information from literature. Although recent Vision-Language Models (VLMs) have emerged as a promising paradigm to automate this complex visual reasoning task, their application is fundamentally bottlenecked by the inability to align visual chemical entities with pre-trained knowledge, alongside the inherent discrepancy between token-level training and reaction-level evaluation. To address these dual challenges, this work enhances VLM-based RxnDP from two complementary perspectives: prompting representation and learning paradigms. First, we propose Identifier as Visual Prompting (IdtVP), which leverages naturally occurring molecule identifiers (e.g., bold numerals like 1a) to activate the chemical knowledge acquired during VLM pre-training. IdtVP enables powerful zero-shot and out-of-distribution capabilities, outperforming existing prompting strategies. Second, to further optimize performance within fine-tuning paradigms, we introduce Re3-DAPO, a reinforcement learning algorithm that leverages verifiable rewards to directly optimize reaction-level metrics, thereby achieving consistent gains over standard supervised fine-tuning. Additionally, we release the ScannedRxn benchmark, comprising scanned historical reaction diagrams with real-world artifacts, to rigorously assess model robustness and out-of-distribution ability. Our contributions advance the accuracy and generalization of VLM-based reaction diagram parsing. We will release data, models, and code on GitHub.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 38 of its 55 references that are in the map. First posted on arXiv 16 Mar 2026.
Builds on
38 of 55 references.
Closest ancestors · 1
Same direction, newest first
- 2026TSSR: Two-Stage Swap-Reward-Driven Reinforcement Learning for Character-Level SMILES Generation8 Jan
Other references · 37
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Visual Instruction Tuningcited by 7,869
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2023Qwen Technical Reportcited by 3,344
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
- 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
- 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 526
25 more
- 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 461
- 2021OCR-free Document Understanding Transformercited by 407
- 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
- 2023OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documentscited by 316
- 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 289
- 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 267
- 2023Nougat: Neural Optical Understanding for Academic Documentscited by 245
- 2023Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Textcited by 208
- 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Documentcited by 174
- 2023ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Promptscited by 161
- 2025Perception-R1: Pioneering Perception Policy with Reinforcement Learningcited by 120
- 2024On Designing Effective RL Reward at Training Time for LLM Reasoningcited by 87
- 2025MM-RLHF: The Next Step Forward in Multimodal LLM Alignmentcited by 83
- 2024Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Modelscited by 65
- 2024OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Textcited by 59
- 2025RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agentscited by 25
- 2023RxnScribe: A Sequence Generation Model for Reaction Diagram Parsingcited by 17
- 2024OpenChemIE: An Information Extraction Toolkit For Chemistry Literaturecited by 17
- 2026FireRed-OCR Technical Reportcited by 13
- 2022Don't Judge a Language Model by Its Last Layer: Contrastive Learning with Layer-Wise Attention Poolingcited by 12
- 2024TAMER: Tree-Aware Transformer for Handwritten Mathematical Expression Recognitioncited by 9
- 2023ReactIE: Enhancing Chemical Reaction Extraction with Weak Supervisioncited by 7
- 2026Agentic reinforcement learning empowers next-generation chemical language models for molecular design and synthesiscited by 6
- 2025Survey on Recent Progress of AI for Chemistry: Methods, Applications, and Opportunitiescited by 3
- 2025RxnCaption: Reformulating Reaction Diagram Parsing as Visual Prompt Guided Captioningcited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Tasks
- 2026MACReD: A Multi-Agent Collaborative Reasoning Framework for Reaction Diagram Parsing
- 2026ChemVLR: Prioritizing Reasoning in Perception for Chemical Vision-Language Understanding
- 2025A Scientific Reasoning Model for Organic Synthesis Procedure Generation
- 2025RxnBench: A Multimodal Benchmark for Evaluating Large Language Models on Chemical Reaction Understanding from Scientific Literature
- 2025GTR-CoT: Graph Traversal as Visual Chain of Thought for Molecular Structure Recognition
- 2025Towards Large-scale Chemical Reaction Image Parsing via a Multimodal Large Language Model
- 2024Text-Augmented Multimodal LLMs for Chemical Reaction Condition Recommendation