ChemVA: Advancing Large Language Models on Chemical Reaction Diagrams Understanding
Not on the map. Why?
Abstract (arXiv)
While Large Language Models (LLMs) have revolutionized scientific text processing, they exhibit a significant capability gap when interpreting chemical reaction diagrams. We identify two fundamental bottlenecks restricting current systems: a Visual Deficit, where generic vision encoders struggle to resolve the strict topological connectivity of dense molecular graphs, and a Semantic Disconnect, where standard linear strings, such as SMILES, fail to effectively activate the model's latent chemical reasoning. To bridge these gaps, we propose the Chemical Visual Activation (ChemVA) framework, which employs a Visual Anchor mechanism to ground functional groups via hybrid-granularity detection, followed by a semantic alignment approach that translates visual features into entity names to maximize knowledge activation in LLMs. We evaluate our approach on OCRD-Bench, a newly constructed dataset featuring dense visual-semantic contexts and comprehensive reaction coverage to evaluate the full spectrum from recognition to reasoning. Extensive experiments on OCRD-Bench demonstrate that ChemVA achieves 92.0% structural recognition accuracy. By bridging visual and semantic bottlenecks, our framework delivers a consistent performance gain of approximately 20 percentage points across 9 diverse LLMs, enabling open-weight models to rival proprietary SOTA systems in complex chemical reasoning tasks.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 51 references that are in the map. First posted on arXiv 17 May 2026.
Builds on
18 of 51 references.
References · 18
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
- 2023What can Large Language Models do in chemistry? A comprehensive benchmark on eight taskscited by 185
- 2024ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Areacited by 71
- 2022MolScribe: Robust Molecular Structure Recognition with Image-To-Graph Generationcited by 49
- 2018Molecular Structure Extraction From Documents Using Deep Learningcited by 25
- 2024OpenChemIE: An Information Extraction Toolkit For Chemistry Literaturecited by 17
- 2025GTR-CoT: Graph Traversal as Visual Chain of Thought for Molecular Structure Recognitioncited by 8
- 2026MolecularIQ: Characterizing Chemical Reasoning Capabilities Through Symbolic Verification on Molecular Graphscited by 8
6 more
- 2024ChemMiner: A Large Language Model Agent System for Chemical Literature Data Miningcited by 7
- 2025oMeBench: Towards Robust Benchmarking of LLMs in Organic Mechanism Elucidation and Reasoningcited by 4
- 2021Self-Labeling of Fully Mediating Representations by Graph Alignmentcited by 3
- 2025Multimodal Search in Chemical Documents and Reactionscited by 2
- 2025MolMole: Molecule Mining from Scientific Literaturecited by 2
- 2025Doc2SAR: A Synergistic Framework for High-Fidelity Extraction of Structure-Activity Relationships from Scientific Documentscited by 2
Built on by
Not cited yet.