# MOF-VERIFY: A Failure-Aware Agentic Harness for MOF Hypothesis Verification

arXiv: https://arxiv.org/abs/2610.03056 · HTML: https://mlsanity.com/paper/2610.03056/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 8 of its 28 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 8 of 8)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ChemCrow: Augmenting large-language models with chemistry tools](https://mlsanity.com/paper/2304.05376/) (2023)
- [Accelerating scientific discovery with Co-Scientist](https://mlsanity.com/paper/2502.18864/) (2025)
- [ChatGPT Chemistry Assistant for Text Mining and Prediction of MOF Synthesis](https://mlsanity.com/paper/2306.11296/) (2023)
- [Disentangling Memory and Reasoning Ability in Large Language Models](https://mlsanity.com/paper/2411.13504/) (2024)
- [A practical guide to machine learning interatomic potentials -- Status and future](https://mlsanity.com/paper/2503.09814/) (2025)
- [SimMOF: AI agent for Automated MOF Simulations](https://mlsanity.com/paper/2603.29152/) (2026)
- [MIND: AI Co-Scientist for Material Research](https://mlsanity.com/paper/2604.13699/) (2026)

## Abstract (arXiv)

Large language models are increasingly used as reasoning components in AI-driven materials Co-Scientists, yet the reliability of the resulting verification pipeline remains unclear. Metal-organic frameworks (MOFs) provide a particularly challenging setting because structures may appear under different identifiers, synthesis outcomes depend strongly on experimental conditions, evidence is distributed across heterogeneous sources, and some hypotheses require computation rather than literature alone. We introduce a diagnostic benchmark with four task families covering structural grounding, synthesis-condition verification, evidence-sufficiency verification, and MLIP-based computational verification. T-MOF-1-3 are evaluated under closed-book, retrieval-enabled, and oracle-evidence settings to localize failures in knowledge access, evidence acquisition, and reasoning, while T-MOF-4 separately evaluates computational verification. Guided by these diagnosed failure modes, we develop MOF-Verify, a failure-aware agentic harness that targets structural, literature, evidence-sufficiency, and computational bottlenecks before producing a final verdict. Across multiple backbone LLMs, MOF-Verify substantially improves hypothesis-verification performance over direct inference and retrieval-based baselines. Benchmark datasets are released at https://github.com/IMMS-Ewha/MOF-Verify-Benchmark.