# VERDICT: Agreement Beats Pixel-Space Verification in Real-Document OCSR

arXiv: https://arxiv.org/abs/2608.22183 · HTML: https://mlsanity.com/paper/2608.22183/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 23 Aug 2026.

## Builds on (most cited first, 11 of 11)

- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control](https://mlsanity.com/paper/2110.01052/) (2021)
- [ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area](https://mlsanity.com/paper/2408.07246/) (2024)
- [MolScribe: Robust Molecular Structure Recognition with Image-To-Graph Generation](https://mlsanity.com/paper/2205.14311/) (2022)
- [Molecular Structure Extraction From Documents Using Deep Learning](https://mlsanity.com/paper/1802.04903/) (2018)
- [RxnScribe: A Sequence Generation Model for Reaction Diagram Parsing](https://mlsanity.com/paper/2305.11845/) (2023)
- [OpenChemIE: An Information Extraction Toolkit For Chemistry Literature](https://mlsanity.com/paper/2404.01462/) (2024)
- [MolNexTR: A Generalized Deep Learning Model for Molecular Image Recognition](https://mlsanity.com/paper/2403.03691/) (2024)
- [Real Data Closes Synthetic-to-Real Gap in Optical Chemical Structure Recognition](https://mlsanity.com/paper/2608.09100/) (2026)

## Abstract (arXiv)

Optical Chemical Structure Recognition (OCSR) converts 2D molecular depictions in the published literature into SMILES, and is increasingly important for constructing large-scale chemical training datasets. Automation at that scale requires identifying unreliable predictions in the absence of ground truth. Three families of label-free signals were compared on 263 ACS journal depictions with verified ground truth: model confidence, re-rendering similarity, and agreement among recognizers. Pixel-space re-rendering performed little better than chance (AUROC 0.547, 95% CI [0.465,0.629]), and an oracle-tuned threshold on it reduced correct labels per image from 0.745 to 0.205. Agreement among four architecturally distinct recognizers instead reached an AUROC of 0.916 ([0.880,0.952]). The two-of-four rule accepted 81.7% of images at 88.8% precision, the three-of-four rule 52.1% at 98.5%. The same pattern held on CLEF-IP, UOB, and USPTO. This distinction is obscured on synthetic benchmarks, where re-rendered predictions naturally resemble their inputs. A substance filter removed 2,193 false agreements on wildcards and R-group fragments, after which the three-of-four rule rejected all 68 generic depictions. VERDICT was then applied to PMC Open Access, producing 6,146 structure labels for 4,833 molecules; chemist adjudication of 400 released labels in two independent samples yielded precisions of 0.995 for the three-of-four tier and 0.958 for the two-of-four tier. VERDICT therefore enables validated labels for multimodal molecular databases linking structure images, machine-readable representations, and source-publication information. In SES AI's Molecular Universe platform, VERDICT further serves as an image-based interface for searching and retrieving molecular records.